ACROSS: An Alignment-based Framework for Low-Resource Many-to-One Cross-Lingual Summarization
Year: 2023

Abstract:
This research addresses the challenges of CrossLingual Summarization (CLS) in low-resource scenarios and over imbalanced multilingual data. Existing CLS studies mostly resort to pipeline frameworks or multi-task methods in bilingual settings. However, they ignore the data imbalance in multilingual scenarios and do not utilize the high-resource monolingual summarization data. In this paper, we propose the Aligned CROSs-lingual Summarization (ACROSS) model to tackle these issues. Our framework aligns lowresource cross-lingual data with high-resource monolingual data via contrastive and consistency loss, which help enrich low-resource information for high-quality summaries. In addition, we introduce a data augmentation method that can select informative monolingual sentences, which facilitates a deep exploration of high-resource information and introduce new information for low-resource languages. Experiments1 on the CrossSum dataset show that ACROSS outperforms baseline models and obtains consistently dominant performance on 45 language pairs.

heading: 1 Introduction
text: Given a source document, Cross-Lingual Summarization (CLS) aims to generate a summary in a different language. Therefore, CLS helps users quickly understand news outlines written in foreign, unknown to them, languages. Early CLS approaches typically use pipeline frameworks (Leuski et al., 2003; Orasan and Chiorean, 2008), which are intuitive but suffer from the problem of error cascading. Researchers have recently turned to endto-end models (Zhu et al., 2019, 2020; Bai et al., 2021) that are immune to this problem. However, these studies are limited to bilingual learning and do not conform to the reality of multilingual scenarios.
∗Corresponding author. 1https://github.com/Youggls/ACROSS-ACL23
Given that the real-world news is written in diverse languages and that only a few researchers have explored the multilingual scenarios, we investigate the many-to-one CLS scenario to meet realistic demands. As stated before, CLS data can be viewed as low-resource since parallel CLS data is significantly less abundant than monolingual data (Zhu et al., 2019). The low-resource characteristic of CLS data is further amplified in multilingual scenarios. However, directly training an end-to-end model does not perform well due to the ineffective use of high-resource data and the scarcity of lowresource data. The foremost challenges are how to model cross-lingual semantic correlations in multilingual scenarios and introduce new knowledge to low-resource languages.
To tackle the above challenges, we investigate a novel yet intuitive idea of cross-lingual alignment.
2458
The cross-lingual alignment method can extract deep semantic relations across languages. As portrayed in Figure 1, the materials in three languages (i.e., French, Chinese, and English) express similar semantics. We can align all these languages for deep cross-lingual semantic knowledge, which is crucial for refining crosslingual materials over different languages for generating high-quality summaries. Moreover, we also consider devising a novel data augmentation (DA) method to introduce new knowledge to low-resource languages.
To investigate the two hypotheses, we introduce a novel many-to-one CLS model for lowresource learning called Aligned CROSs-lingual Summarization (ACROSS), which improves the performance of low-resource scenarios by effectively utilizing the abundant high-resource data. This model conducts cross-lingual alignments both at the model and at the data levels. From the model perspective, we minimize the difference between the cross-lingual and monolingual representations via contrastive and consistency learning (He et al., 2020; Pan et al., 2021; Li et al., 2021, 2022). This helps to facilitate a solid alignment relationship between low-resource and high-resource language. From the data perspective, we propose a novel data augmentation method that selects informative sentences from monolingual summarization (MLS) pairs, which aims to introduce new knowledge for low-resource language.
We conducted experiments on the CrossSum dataset (Hasan et al., 2021), which contains crosslingual summarization pairs in 45 languages. The results show that ACROSS outperforms the baseline models and achieves strong improvements in most language pairs (2.3 average improvement in ROUGE scores).
Our contributions are as follows:
• We propose a novel many-to-one summarization model that aligns cross-lingual and monolingual representations to enrich low-resource data.
• We introduce a data augmentation method to extract high-resource knowledge which is later transferred and which facilitates lowresource learning.
• An extensive experimental evaluation validate the low-resource CLS performance of our model in both quantitative and qualitative ways.

heading: 2 Related Work
text: Early CLS research typically used pipeline methods, such as the translate-then-summarize (Leuski et al., 2003; Ouyang et al., 2019) or summarizethen-translate methods (Orasan and Chiorean, 2008; Wan et al., 2010; Yao et al., 2015; Zhang et al., 2016), which are sensitive to error cascading that causes their subpar performance.
Thanks to the development of the transformerbased methods (Vaswani et al., 2017), researchers introduced teacher-student frameworks (Shen et al., 2018; Duan et al., 2019) wherein the CLS task can be approached via an encoder-decoder model. Thereafter, the multi-task framework started to be popular in this field (Zhu et al., 2019, 2020; Bai et al., 2021). Recently, researchers have begun to investigate how to fuse translation and summarization tasks into a unified model to improve the performance on the CLS tasks (Liang et al., 2022; Takase and Okazaki, 2022; Bai et al., 2022; Nguyen and Luu, 2022; Jiang et al., 2022). For example, Bai et al. (2022) considered compression so that their model can handle both the CLS and translation tasks at different compression rates.
Focusing on multi-task learning, these multi-task studies attempt to improve CLS performance using machine translation (MT) and MLS tasks in bilingual settings. However, such approaches still establish implicit connections among languages and leave aside the information of high-resource data.
Hasan et al. (2021) recognized the limitations of the above-mentioned scenarios. They proposed a new dataset, CrossSum, in multilingual scenarios and introduced a method balancing the number of different language pairs in a batch, which could alleviate the uneven distribution of training samples and balance performance in different languages. However, deep semantical correlations across languages as well as abundant information from highresource data have not been investigated.
In contrast to the aforementioned methods, ACROSS introduces cross-lingual alignment and a novel data augmentation method, which can improve low-resource performance from both model and data perspectives.

heading: 3 Aligned CROSs-lingual Summarization
text: In this section, we explain the details of ACROSS. ACROSS introduces alignment constraints at both
the encoder and decoder sides. Figure 2 illustrates the overall framework.

heading: 3.1 Preliminary
text: Mono-Lingual Abstractive Summarization. Given a document DA = {xA1 , xA2 , ..., xAn } written in language A, a monolingual abstractive summarization model induces a summary SA = {yA1 , yA2 , ..., yAm} by minimizing the loss function as follows:
Labs = − n∑
t=1
logP (yAt |yA<t, DA,θmls), (1)
where n and m are the lengths of the input document and output summary, respectively, and θmls is the parameter of the monolingual summarization model.
Cross-Lingual Abstractive Summarization. Different from monolingual abstractive summarization models, a cross-lingual abstractive summarization model generates a summary SB = {yB1 , yB2 , ..., yBm} in language B when given a source document DA = {xA1 , xA2 , ..., xAn } in language A. The loss function of the CLS model can be formulated as:
Lcls = − n∑
t=1
logP (yBt |yB<t, DA,θcls), (2)
where θcls is the parameter of the CLS model.

heading: 3.2 Cross-Lingual Alignment
text: Cross-Lingual Contrastive Learning for Encoder. Multilingual transformer treats all languages equally, which leads to the representation of
different languages being distributed over different spaces, eventually making it difficult for CLS tasks to take advantage of the high-resource monolingual data. Therefore, we should encourage the model to improve cross-lingual performance with a strong monolingual summarization capability. With the help of contrastive learning, ACROSS can align the cross-lingual input representation to the monolingual space, thus realizing the idea mentioned above.
Firstly, given a cross-lingual summarization and the paired monolingual document tuple: (DA, DB+ , S
B), we need to randomly choose a negative document set N = {DB1 , DB2 , ..., DB|N |} in the dataset. Then, we can obtain the representation of DA with a Transformer Encoder and a pooling function F as follows:
hA = F(Encodercls(DA)). (3) Similarly, we can obtain the representation of DB+ with a pretrained Encoder of the monolingual summarization model as:
hB = F(Encoder∗mls(DB)). (4) Finally, the contrastive learning objective is con-
structed to minimize the loss as follows:
Lctr = − log esim(h A,hB+)/τ
∑ i∈idx(N ) e sim(hA,hBi )/τ , (5)
where τ is a temperature hyper-parameter and sim(·) denotes a similarity function that can measure the distance of two vectors in an embedding space2.
2We use cosine similarity as the similarity function.
Cross-Lingual Consistency Learning for Decoder. Consistency learning aims to model consistency across the models’ predictions, which can help child models gain improvement from the pretrained parent model. By constraining the output probability distributions of decoders, the CLS child model can be aligned to the MLS pre-trained parent model.
Given a tuple composed of a CLS document and its paired monolingual document (DA, DB+), we can obtain the output distribution of the CLS model at each decoding step as follows:
P (yBt |yB<t, DA,θcls) = Modelcls(yB<t, DA). (6)
Similarly, we can construct the output distribution of the MLS model at each decoding step as:
P (yBt |yB<t, DB+ ,θ∗mls) = Model∗mls(yB<t, DB+), (7) where θ∗mls denotes frozen parameters and Model∗mls means that the parameters of the MLS model are frozen during training. Then, we can bridge the distribution gap between the CLS and MLS models by minimizing the following consistency loss function as:
Lcon = n∑
t=1
JS-Div[P (yBt |yB<t, DA,θcls),
P (yBt |yB<t, DB+ ,θ∗mls)], (8)
where JS-Div denotes Jensen–Shannon divergence (Lin, 1991), which is used to measure the gap between the pretrained and child models.
Training Objective of ACROSS. We jointly minimize CLS, consistency, and contrastive loss during the training period. The final training objective of ACROSS is formulated as:
L = α · Lcls + β · Lctr + γ · Lcon, (9)
where α, β, and γ are hyper-parameters used to balance the weights of the three losses.

heading: 3.3 Data Augmentation for Cross-Lingual Summarization
text: Data augmentation is a widely used technique in low-resource scenarios (Sennrich et al., 2016a; Fabbri et al., 2021). In Seq2Seq tasks, it often leverages translation to increase the amount of data in low-resource scenarios. However, in the CLS task, the direct translation of monolingual data from a
high-resource language to a low-resource language might lose some valuable information. The distribution of information in the input document is uneven, making some sentences potentially more important than others. Therefore, directly translating all sentences into a low-resource language and using them for training the model may not be conducive to CLS.
Considering the characteristics of the summarization task, we propose an importance-based data augmentation method based on ROUGE scores. First, an input document DBi is split into several sentences S = {s1, s2, ..., sk}. Then, the ROUGE score is calculated for each sentence and summary SB . The ROUGE score of each sentence is represented as R = {r1, r2, ..., rk}. Next, the sentences corresponding to the top a% ROUGE scores are selected and translated into the low-resource language. Finally, the translated sentences are reassembled with other sentences to form a pseudo document DApi , so that pseudo-low-resource summarization pairs (DApi , S
B) can be generated. Figure 3 shows an example of the process from an English monolingual summarization pair to a Chinese-English summarization pair3. The two
3We set a=50.
sentences with the highest ROUGE scores are the second and third sentences; hence, these two sentences are translated into Chinese.

heading: 4 Experiment Setup

heading: 4.1 Dataset
text: We conduct experiments using the previously mentioned CrossSum dataset (Hasan et al., 2021). CrossSum is a multilingual CLS dataset that contains cross-lingual summarization data in 45 languages. Moreover, it realistically reflects the skewness of data distribution in practical CLS tasks. Figure 4 portrays the degree of imbalance of the dataset. As we can see, English monolingual summaries constitute over 70% of the English target summaries, while there are less than 30% summaries of other 44 languages to English. We classify languages with less than 1,000 training samples as extremely low-resource scenarios, between 1,000 and 5,000 as medium low-resource scenarios, and larger than 5,000 as normal low-resource scenarios.

heading: 4.2 Baselines
text: We compare our model with the following baselines:
Multistage: a training sampling strategy proposed by Hasan et al. (2021). Thi s method balances the number of different language pairs in a batch, thus alleviating the uneven distribution of training samples in different languages.
NCLS+MT: a method based on the multi-task framework proposed by Zhu et al. (2019). The model uses two independent decoders for CLS and MT tasks. As the original NCLS+MT model can only handle bilingual CLS task, we replace its encoder with a multilingual encoder.
NCLS+MLS: a method also proposed by Zhu et al. (2019). Its difference with NCLS+MT is that the multi-task decoder is used for MLS task.

heading: 4.3 Experimental Settings
text: For training, the MLS model is trained on the English-English subset of the CrossSum dataset and its parameters are initialized using mT5 (Xue et al., 2021). Thereafter, we initialize the CLS model using the pre-trained MLS model. We set dropout to 0.1 and the learning rate to 5e-4 with polynomial decay scheduling as well as a warm-up step of 5,000. For optimization, we use the Adam optimizer (Kingma and Ba, 2015) with ϵ = 1e-8, β1 = 0.9, β2 = 0.999, and weight decay = 0.01. The hyper-parameters α, β, and γ are set to 1.0, 1.0, and 2.0, respectively. The size of the negative sample set is 1,024. The temperature hyper-parameter τ is set to 0.1. To stabilize the training process, we choose the gradient norm value to be 1.0. The vocabulary size is 250,112, and BPE (Sennrich et al., 2016b) is used as the tokenization strategy. We limit the max input length to 512 and the max summary length to 84. We train our model on 4 RTX A5000 GPUs for 40,000 training steps, setting the batch size to 256 for each step. For inference, we use the beam-search decoding strategy (Wiseman and Rush, 2016) and set the beam size to 5.

heading: 5 Experiment Results

heading: 5.1 Main Results
text: We evaluate ACROSS on the standard ROUGE metric (Lin, 2004), reporting the F1 score (%) of ROUGE-1, ROUGE-2, and ROUGE-L. Table 1 presents the main results of ACROSS and other
models on different low-resource settings. base and small refer to different mT5 settings. base model contains a 12-layer encoder and 12-layer decoder with 768-dimensional hidden representations. small model contains an 8-layer encoder and 8-layer decoder with 512-dimensional hidden representations. As discussed in Section 4.1, we classify languages as extremely, medium and normal low-resource scenarios. It should be clarified that although we have artificially divided languages into different low-resource scenarios, any CLS language pair is actually low-resource compared to the English-English data volume.
Comparision with Multistage. Compared with the Multistage-base, ACROSS-base obtains 1.95, 2.45, and 2.17 ROUGE-2 improvements for extremely, medium and normal low-resource scenarios, respectively. Furthermore, ACROSS-base reaches 3.01, 3.76, and 2.95 ROUGE-1 improvements for extremely, medium and normal lowresource scenarios, respectively. The ROUGE-L scores for extremely, medium and normal lowresource scenarios are also improved by 2.32, 3.2, and 2.65, respectively. As shown in Figure 5, ACROSS-base outperforms Multistage-base significantly under the different language test sets. The ROUGE2 scores for more than 30 languages have an increase of more than 2, which represents a stable improvement of ACROSS.
Moreover, ACROSS-small surpasses or is compared to Multistage-base under some metrics (improving ROUGE-1 by 0.2 and ROUGE-L by 0.09 in extremely low-resource scenarios).
In addition, we can see in Figure 5 that EnglishEnglish’s ROUGE-2 score improves by only 0.77, which illustrates that the improvement of ACROSS comes mainly from the better alignment between other languages and English, rather than from
the improvement of the ability to do summarization on English. Considering the actual data size, ACROSS significantly overperforms baselines in low-resource CLS scenarios. Additionally, we demonstrate ROUGE-1 and ROUGE-L results in Appendix A.
Comparision with Multi-Task Methods. Compared with the two multi-task methods (i.e., NCLS+MT and NCLS+MLS), we find that the two methods do not perform as well as Multistage and have a greater gap with ACROSS. Compared with NCLS+MT and NCLS+MLS, the ROUGE-
1, ROUGE-2, and ROUGE-L scores of ACROSS are enhanced by more than 3, 1, and 2, respectively. This phenomenon reveals that multi-task approaches that rely on MT and MLS learning may be not effective in multilingual scenarios. ACROSS turns to be more suitable for the scenarios with imbalanced resources.

heading: 5.2 Analysis
text: Ablation Study. We next conduct the ablation study in small settings. We summarize the experimental results in Table 2 as below:
• ctr+con+DA performs better than con+DA, suggesting that although con can significantly improve performance, the aligned representation is also beneficial for CLS tasks.
• The complete model produces better results compared with DA. Except for Multistage, DA performs worse than the models adding other losses, which implies that the excellent performance of ACROSS does not merely come from data augmentation.
• Comparing DA and con, we can see that the aligned model and representation are crucial for a successful CLS task.
Analysis of Data Augmentation. We conduct experiments on different selection approaches to evaluate the performance of our proposed DA method.
As recorded in Table 3, Informative performs best compared to the other methods, which indicates that the DA method can help ACROSS learn more important information in the CLS task. The Truncation performs inferior, because the more important sentences in the news report tend to be in
the relatively front position. The results also validate the effectiveness of the DA method in selecting more important sentences for translation.
Generally speaking, the results tell us that the DA method is beneficial for the CLS task, and translating important sentences is useful for crosslingual alignment.
Human Evaluation. Due to the difficulty of finding a large number of users who speak low-resource languages, we only conduct the human evaluation on 20 random samples from Chinese-English and French-English test sets. We compare the summaries generated by ACROSS with those generated by Multistage. We invite participants to compare the auto-generated summaries with ground truth summaries from three perspectives: fluency (FL), informativeness (IF), and conciseness (CC). Each sample is evaluated by three participants.
The results shown in Table 4 indicate that ACROSS is capable of generating fluent summaries and these summaries are also informative and concise according to the human feelings.
Visualization of Alignment. To further demonstrate the alignment result of ACROSS, we visualize the similarity between CLS inputs and the paired English inputs in Chinese-English and French-English test sets. We randomly sample 50 cross-lingual inputs from the test set and obtain the
representations of these cross-lingual inputs and the paired English inputs. Then, we calculate the cosine similarity of the two languages to construct the similarity matrix. Finally, we plot the heat map of the similarity matrix.
In Figure 6b and Figure 6d, the clear diagonal indicates the paired inputs have significantly higher similarities. In comparison, other unpaired inputs have lower similarities. In Figure 6a and Figure 6c, we can observe that the similarity distribution is characterized by more confusion.
In summary, ACROSS can effectively align cross-lingual and English inputs, demonstrating through the experiments that aligned representations are more useful for CLS tasks in multilingual settings.
Case Study. We finally implement the case study of a sample from the Chinese-English test set. The Baseline employed here is the Multistage model. The words and characters in red are important and overlap with Ground Truth. On the opposite, the words in green are errors. As shown in Figure 7, compared to Multistage, ACROSS can cover details in a better and more detailed way (e.g., using some proper nouns and phrases). For example, asthma and processed meat are present in the generated summary by ACROSS; yet, the summary generated by the baseline does not involve these important terms, and it also contains factual consistency errors. Taking another example, in the summary generated by the baseline, the terms fruit and vegetables, including cabbage, broccoli, and kale appear, while these terms are not mentioned in the original text.
The above examples suggest that ACROSS improves the performance of CLS based on the ability
of strong MLS under the guidance of alignment.

heading: 6 Conclusion
text: In this work, we propose ACROSS, a many-toone cross-lingual summarization model. Inspired by the alignment idea, we design contrastive and consistency loss for ACROSS. Experimental results show that with the ACROSS framework, CLS model improves the low-resource performance by effectively utilizing high-resource monolingual data. Our findings point to the importance of alignment in cross-lingual fields for future research. In
the future, we plan to apply this idea to combine CLS in multimodal scenarios, which might enable the model to better serve realistic demands.

heading: Acknowledgements
text: This work was supported in part by National Natural Science Foundation of China under Grant No. 62106091, and in part by the Shandong Provincial Natural Science Foundation under Grant No. ZR2019MF062.
Limitations
Considering that English is the most widely spoken language, we select it as the high-resource monolingual language in this study. While ACROSS is a general summarization framework not limited to a certain target language, it deserves an in-depth exploration of how ACROSS works on other highresource languages.
Additionally, we employ mT5 as our backbone because it supports most languages in CrossSum. The performance of ACROSS after replacing mT5 with other models, such as mBART(Liu et al., 2020), FLAN-T5(Chung et al., 2022), will be investigated in the future.
Ethical Consideration
Controversial Generation Content. Our model is less likely to generate controversial content(e.g., discrimination, criticism, and antagonism) since the model is trained on a dataset from the BBC News domain. Data in the news domain is often scrutinized before being published, and thus the model is not likely to generate controversial data.
Desensitization of User Data. We use the Amazon Mechanical Turk crowdsourcing platform to evaluate three artificial indicators (i.e., fluency, informativeness, and conciseness). For investigators, all sensitive user data is desensitized by the platform. Therefore, we also do not have access to sensitive user information.

heading: A Appendix
text: Analysis of Alignment Methods. To further show the effectiveness of ACROSS, we conduct an experiment to analyze the alignment methods. We replace the alignment methods of the encoder and decoder. As Table 5 shows, replacing any part of the original alignment methods will make the model perform worse. In particular, replacing the consistency and contrastive loss at the same time significantly reduces the model’s performance, which reinforces the rationality of our different loss designs.
Data Augmentation Settings. We use HelsinkiNLP4 as our translation model. In practice, we select the sentences corresponding to the top 50% of ROUGE scores. Furthermore, we set the beam size to 4, lenght-penalty to 1.0 and min-length to 10 for decoding.
ROUGE-1 & ROUGE-L Improvement for ACROSS-base. To show the improvement of our model on different metrics, we plot the improvement compared to Multistage-base, similar to Figure 5. As Figure 8 and 9 shows, ACROSSbase also has a significant and stable improvement on ROUGE-1 and ROUGE-L among different languages.
Analysis of Translation Ratio. We also analyze the impact of the translation ratio α on the final results. As table 6 shows, ACROSS-100% performs worse instead, probably because translating all sentences introduces too much extraneous noise instead.
Improvement in Different Resource Scenarios. We also analyze the improvement in different resource scenarios. As table 7 shows, low-resource
4https://huggingface.co/Helsinki-NLP
0 1 2 3 4 5 6 7 8 Rouge1-F
ig fr pcm az
sr_cy am ne ha ja rn ko bn
ti vi th ar sr_la so uk hi tr fa yo ur pt ta id
sw ru pa zh_CN cy te zh_TW gu mr es ps
my si ky en gd uz
+7.99 +5.60
+5.25 +4.87 +4.85 +4.82
+4.69 +4.64
+4.55 +4.53
+4.23 +4.16 +4.10 +4.08 +4.08
+3.96 +3.88
+3.66 +3.61 +3.57 +3.56 +3.50 +3.43 +3.38 +3.33 +3.28 +3.21
+3.11 +3.05 +3.01 +3.01
+2.92 +2.78
+2.44 +2.42
+2.22 +2.20
+2.10 +1.98
+1.66 +1.64
+1.25 +0.59
+0.27
Figure 8: The improvement of ACROSS-base compared to Multistage-base in ROUGE1-F on different languages to English.
languages get a more significant improvement compared to high-resource languages.
Form for Human Evaluation. Figure 10 shows the form we gave to participants, on the case of French-English summarization evaluation. Participants were asked to compare the auto-generated summaries with ground truth summaries from three perspectives: fluency, informativeness, and conciseness from one to five. And each participant will be informed that their scores for the different summaries will appear in our study as an evaluation metric.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
7
7 A2. Did you discuss any potential risks of your work? We report a series of experimental setups in the paper, including model size, experimental data, performance on different languages, etc. Our proposed approach is also a generic generalization approach that takes into account universal scenarios.
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Left blank.
3 B1. Did you cite the creators of artifacts you used? 2
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The dataset we use is released under license CC BY-NC-SA 4.0. The license is restricted only to those who want to modify and redistribute it, who need to use the same license as it.
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 4
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? 8
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 4
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 4
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? 5
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 4
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 4
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 5
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? 5
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? appendix a
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? We used the platform provided by Amazon for human evaluation and charged 0.02$ per piece of data, which is also in line with the price of most text tasks.
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? appendix a
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Not applicable. Left blank.

heading: Future Work
text: , 2022), will be investigated in the future. Ethical Consideration
Controversial Generation Content. Our model is less likely to generate controversial content(e. g. , discrimination, criticism, and antagonism) since the model is trained on a dataset from the BBC News domain. Data in the news domain is often scrutinized before being published, and thus the model is not likely to generate controversial data. Desensitization of User Data. We use the Amazon Mechanical Turk crowdsourcing platform to evaluate three artificial indicators (i. e. , fluency, informativeness, and conciseness). For investigators, all sensitive user data is desensitized by the platform. Therefore, we also do not have access to sensitive user information