Sci- L o RA : Mixture of Scientific L o RA s for Cross-Domain Lay Paraphrasing (2025)

Abstract: Lay paraphrasing aims to make scientific information accessible to audiences without technical backgrounds. However, most existing studies focus on a single domain, such as biomedicine. With the rise of interdisciplinary research, it is increasingly necessary to comprehend knowledge spanning multiple technical fields. To address this, we propose Sci-LoRA, a model that leverages a mixture of LoRAs fine-tuned on multiple scientific domains. In particular, Sci-LoRA dynamically generates and applies weights for each LoRA, enabling it to adjust the impact of different domains based on the input text, without requiring explicit domain labels. To balance domain-specific knowledge and generalization across various domains, Sci-LoRA integrates information at both the data and model levels. This dynamic fusion enhances the adaptability and performance across various domains. Experimental results across twelve domains on five public datasets show that Sci-LoRA significantly outperforms state-of-the-art large language models and demonstrates flexible generalization and adaptability in cross-domain lay paraphrasing.

## introduction lay paraphrasing aims at making the technical or specialist text comprehensible for non-expert audiences. in an era of abundant specialized knolwedge, lay paraphrasing plays a crucial role in making intricate scientific information and concepts accessible and understandable to people lack of technical expertise, fostering public understanding of science and its impact. with the growing prevalence of interdisciplinary research and collaboration, scientific contents from diverse domains is increasingly reaching the general public. it is particularly important to make the interdisciplinary information comprehensible to the cross-domain nonexpert audience for minimizing misunderstandings and fostering effective cross-domain collaboration. however, existing works on lay paraphrasing are limited to a single domain, such as biomedicine @xcite , scientific news @xcite , etc. though the most recent study has expanded the scope of lay paraphrasing to translate technical language into general-audience language across multiple domains @xcite , the models are still only fine-tuned on separate domain-specific data, assuming domain knowledge is available during inference and not fully leveraging the crossdomain knowledge. these studies focus on developing one model within a specified domain while overlooking the generalization of the model across multiple domains, resulting in potential misinterpretations of cross-domain concepts. besides, it is hard for those models to adapt to unseen information when new scientific interdisciplinary fields emerge. they often require full-scale retraining, which is inefficient and time-consuming. to address the above challenges, this paper explores the "mixed-domain scenario", where scientific content may span one or multiple domains. we introduce sci-lora, a model that leverages a mixture of loras fine-tuned on a diverse set of scientific domains. unlike conventional models that are fine-tuned for one specific domain, sci-lora adopts a multi-lora serving architecture, enabling continuous improvement as new lora modules can be added and updated for newly emerging domains. to effectively utilize cross-domain knowl- ## conclusion in this paper, we propose sci-lora foot_1 , a mixture of scientific loras designed for cross-domain lay paraphrasing. without requiring domainspecific information during inference, sci-lora dynamically generates weights for domain-specific adapters. to enhance domain representation learning, a customized text encoder is fine-tuned using contrastive learning. sci-lora then integrates these domain-specific adapters through a dynamic lora fusion module to facilitate cross-domain text generation. we evaluate sci-lora against multiple state-of-the-art baselines across 12 different domains and 5 distinct datasets, using 10 automatic evaluation metrics as well as human assessment. extensive experimental results demonstrate that sci-lora consistently outperforms existing sota models in the cross-domain lay paraphrasing task.