Cross-lingual Science Journalism: Select, Simplify and Rewrite Summaries for Non-expert Readers (2023)

Abstract: Automating Cross-lingual Science Journalism (CSJ) aims to generate popular science summaries from English scientific texts for non-expert readers in their local language. We introduce CSJ as a downstream task of text simplification and cross-lingual scientific summarization to facilitate science journalists’ work. We analyze the performance of possible existing solutions as baselines for the CSJ task. Based on these findings, we propose to combine the three components - SELECT, SIMPLIFY and REWRITE (SSR) to produce cross-lingual simplified science summaries for non-expert readers. Our empirical evaluation on the Wikipedia dataset shows that SSR significantly outperforms the baselines for the CSJ task and can serve as a strong baseline for future work. We also perform an ablation study investigating the impact of individual components of SSR. Further, we analyze the performance of SSR on a high-quality, real-world CSJ dataset with human evaluation and in-depth analysis, demonstrating the superior performance of SSR for CSJ.

## introduction cross-lingual science journalism (csj) produces popular science stories 1 from english scientific texts for non-expert readers in their local language. csj focuses on simultaneously reducing linguistic complexity and length of the original text. automating csj can facilitate science journalists in their work for writing popular science stories. a real-world example of csj is spektrum der wissenschaft (spektrum) 2 . it is a popular monthly science magazine, the german version of scientific american. the magazine is considered a bridge between scientific literature and the non-expert public. our work is initiated by a collaboration with spek1 we quote stories and summaries interchangeably in this work. 2 https://www.spektrum.de/ target: ein mädchen aus südafrika lebt seit neun jahren mit einer hiv-infektion, ohne den erreger mit medikamenten kontrollieren zu müssen. das berichteten fachleute um avy violari von der university of the witwatersrand in johannesburg auf einer konferenz in paris. [...] translation: a girl from south africa has been living with an hiv infection for nine years without having to control the pathogen with medication. this is what experts around avy violari from the university of the witwatersrand in johannesburg reported at a conference in paris. @xcite mbart: das kind, das fiziert wird mit dem hi-virus bei der geburt, ist die geburtsache für das infizieren des virus bei der geburt. die infektion des ungeborenen kindes mit dem hiv wird als latente hiv-infektion bezeichnet. die ursache für die erkrankung ist noch ungeklärt. translation: the child who @xmath0. the infection of the unborn child with hiv is referred to as latent hiv infection. the cause of the disease is still unclear. ssr: ein neunjähriges kind, das ohne behandlung mit einer antiretroviralen therapie die welt retten kann, ist eine person, die eine hiv-infektion überlebt hat. das berichtet eine arbeitsgruppe um avy violari in der fachzeitschrift proceedings of the national allergy and infectious diseases institute. [...] translation: a nine-year-old child who can save the world without treatment with antiretroviral therapy is a person who survived hiv infection. this is reported by a working group led by avy violari in the specialist journal proceedings of the national allergy and infectious diseases institute. @xcite source fragment: a nine-year-old infected with hiv at birth has spent most of their life without needing any treatment, say doctors in south africa. the child, whose identity is being protected, was given a burst of treatment shortly after birth. they have since been off drugs for eight-and-a-half years without symptoms or signs of active virus. @xcite a few researchers have studied monolingual science journalism (msj) @xcite as a summarization task. in summarization, some efforts have also been made towards monolingual @xcite @xcite and crosslingual @xcite scientific summarization. our preliminary investigation also adopts existing cross-lingual summarization (cls) models to explore csj following the msj's steps. since these models focus only on summary generation, these summaries still need to be simplified for non-expert readers. therefore, we propose csj as a downstream task of text simplification and cross-lingual scientific summarization to generate a coherent cross-lingual popular science story. we analyze the workflow of spektrum's journalists to develop a solution for the csj task. they read complex english scientific articles and mark the essential facts, make them straightforward for non-expert readers, and then write a coherent story in german. influenced by this, we propose to combine the three components -select, simplify and rewrite (ssr) for exploring csj. we follow the divide-and-conquer approach to design ssr so that each component is responsible for only one task. it makes ssr manageable, flexible and innovative as we can train individual components and modify/replace them without affecting the ssr's information flow. contributions 1. we introduce cross-lingual science journalism (csj) as a downstream task of crosslingual scientific summarization and text simplification targeting non-expert readers. 2. to solve csj, we develop a pipeline comprising the three components -select, sim-plify and rewrite (ssr) for producing popular german summaries from english scientific texts. 3. we empirically evaluate the performance of ssr against several existing cls models on the wikipedia dataset with various evaluation metrics. we also analyze ablated ssr models to examine the significance of each component. 4. we evaluate ssr's performance on the spek-trum dataset with human judgments and various statistical features to analyze them linguistically. 2 related work 2.1 science journalism louis and nenkova (2013a,b) investigate msj for the writing quality of new york times science stories by dividing them into three coarse levels of writing quality: clear, interesting and beautiful or well-structured. they also analyze general features of discourse organization and sentence structure. barel-ben david et al. (2020) examine the public's interactions with scientific news written by earlycareer scientists by capturing various features. the authors collect a dataset of 150 science news written by 50 scientists from two websites: mako and ynet. dangovski et al. (2021) consider msj as abstractive summarization and story generation. they collect scientific papers and science daily press releases and apply sequence-to-sequence (s2s) models for generating summaries. these studies are limited in their scope and consider only monolingual texts, thus cannot be used for csj. ## experiments 4.1 datasets wikipedia is collected from the wikipedia science portal for english-german science articles @xcite . it consists of monolingual and cross-lingual parts. we use only the cross-lingual part of this dataset. it contains 50,132 english articles (1572 words) paired with german summaries (100 words). spektrum is a high-quality real-world dataset collected from spektrum der wissenschaft @xcite . it covers various topics in diverse science fields: astronomy, biology, chemistry, archaeology, mathematics, physics, etc. it has 1510 english articles (2337 words) and german summaries (361 words). we use wikipedia with a split of 80-10-10 for experiments, while spektrum is used for zero-shot adaptability as a case study. ## wikipedia results all the results are the average of five runs for each model. we report the f-score of rouge and bs, and fre of all models on wikipedia in for fine-tuned models in comparing the performance of the best baseline with our model from these results indicate a significant improvement in performance. ## spektrum results ## conclusions we propose to study cross-lingual science journalism (csj) as a downstream task of text simplification and cross-lingual scientific summarization. automating csj aims to produce popular crosslingual summaries of english scientific texts for non-expert readers. we develop a pipeline comprising the three components -select, simplify and rewrite (ssr) as a benchmark for csj. our empirical evaluation shows that ssr outperforms all baselines by wide margins on wikipedia and achieves good performance on spektrum. we further explore the ablated models with component replacements, demonstrating the resilience and robustness of the ssr application. we conduct a human evaluation of the spektrum outputs, indicating its good linguistic properties, further affirmed by readability analysis. we plan for joint training of simplify and rewrite models for csj as future work.