Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing
Authors: Ming Cheng, Jiaying Gong, Hoda Eldardiry
Accepted at: acl25

Sci-LoRA: Mixture of Scientific LoRAs for Cross-Domain Lay Paraphrasing

Ming Cheng, Jiaying Gong, Hoda Eldardiry
Virginia Tech
{ming98,gjiaying,hdardiry}@vt.edu

Abstract
Lay paraphrasing aims to make scientific information accessible to audiences without technical backgrounds. However, most existing studies focus on a single domain, such as biomedicine. With the rise of interdisciplinary research, it is increasingly necessary to comprehend knowledge spanning multiple technical fields. To address this, we propose Sci-LoRA, a model that leverages a mixture of LoRAs fine-tuned on multiple scientific domains. In particular, Sci-LoRA dynamically generates and applies weights for each LoRA, enabling it to adjust the impact of different domains based on the input text, without requiring explicit domain labels. To balance domain-specific knowledge and generalization across various domains, Sci-LoRA integrates information at both the data and model levels. This dynamic fusion enhances the adaptability and performance across various domains. Experimental results across twelve domains on five public datasets show that Sci-LoRA significantly outperforms state-of-the-art large language models and demonstrates flexible generalization and adaptability in cross-domain lay paraphrasing.

1 Introduction
Lay paraphrasing aims at making the technical or specialist text comprehensible for non-expert audiences. In an era of abundant specialized knowledge, lay paraphrasing plays a crucial role in making intricate scientific information and concepts accessible and understandable to people lack of technical expertise, fostering public understanding of science and its impact. With the growing prevalence of interdisciplinary research and collaboration, scientific contents from diverse domains is increasingly reaching the general public. It is particularly important to make the interdisciplinary information comprehensible to the cross-domain non-expert audience for minimizing misunderstandings and fostering effective cross-domain collaboration.

Table 1 presents examples of interdisciplinary research content along with their layman-friendly paraphrased versions. For example, the term "influenza A/H1N1 hemagglutinin (HA)" can be simplified to "flu virus" for audiences without a biology background. Similarly, the mathematical expression of a graph, "\(G = (V, E)\)", where \(V\) represents vertices, can be rephrased as "each point represents a version of the virus" for those unfamiliar with computer science concepts.

However, existing works on lay paraphrasing are limited to a single domain, such as biomedicine (Guo et al., 2024; Fonseca and Cohen, 2024), scientific news (Liu et al., 2024c), etc. Though the most recent study has expanded the scope of lay paraphrasing to translate technical language into general-audience language across multiple domains (Cheng et al., 2025), the models are still only fine-tuned on separate domain-specific data, assuming domain knowledge is available during inference and not fully leveraging the cross-domain knowledge. These studies focus on developing one model within a specified domain while overlooking the generalization of the model across multiple domains, resulting in potential misinterpretations of cross-domain concepts. Besides, it is hard for those models to adapt to unseen information when new scientific interdisciplinary fields emerge. They often require full-scale retraining, which is inefficient and time-consuming.

To address the above challenges, this paper explores the "mixed-domain scenario", where scientific content may span one or multiple domains. We introduce Sci-LoRA, a model that leverages a mixture of LoRAs fine-tuned on a diverse set of scientific domains. Unlike conventional models that are fine-tuned for one specific domain, Sci-LoRA adopts a multi-LoRA serving architecture, enabling continuous improvement as new LoRA modules can be added and updated for newly emerging domains. To effectively utilize cross-domain knowl-
Table 1: Examples of interdisciplinary scientific contents and lay paraphrasing for the general audience.

<table>
  <tr>
    <th>Technical Contents</th>
    <th>Lay Paraphrasing</th>
  </tr>
  <tr>
    <td><b>CS + Art History:</b> We propose a four-step human-AI collaboration workflow to support the discovery and clustering of these backdrops. Focusing on the painted backdrops of the American Civil War, we present Backdrop Explorer, a content-based image retrieval (CBIR) system incorporating computer vision and novel user interactions.</td>
    <td>We developed a system called Backdrop Explorer to help people find and organize images of painted backdrops from the American Civil War. It uses advanced computer technology to analyze and group similar images. The process involves four steps where humans and artificial intelligence work together to make searching easier and more interactive.</td>
  </tr>
  <tr>
    <td><b>CS + Computational Biology:</b> We suggest representing antigenic drifts within influenza A/H1N1 hemagglutinin (HA) protein as a graph, \( G = (V, E) \), where \( V \) is the set of vertices representing each possible sequence and \( E \) is the set of edges representing single amino acid substitutions.</td>
    <td>We suggest using a network-like diagram to track how the flu virus changes over time. In this diagram, each point represents a version of the virus, and lines between them show small changes in the virus’s building blocks.</td>
  </tr>
  <tr>
    <td><b>CS + Chemistry:</b> Constructing a robust deep learning model for assessing materials’ structure-property relationships remains a non-trivial task due to highly flexible model architecture and the challenge of selecting appropriate material representation methods. In this regard, we develop advanced deep-learning models and implement them for predicting the quantum-chemical calculated properties (i.e., formation energy) for an enormous number of crystal systems.</td>
    <td>Building a strong deep learning model to understand the relationship between the structure of materials and their properties is still a difficult task. This is because the models can be very complex, and it’s not always easy to choose the best way to represent the materials. To tackle this, we’ve developed advanced deep learning models that can predict certain properties of materials, like formation energy, for a large number of crystal systems.</td>
  </tr>
</table>

edge, an adapter weight generator is designed to dynamically generate and assign weights to each LoRA, adjusting the influence based on the relevance of different domains. Specifically, a text encoder is trained using contrastive learning to better distinguish representations across domains. Then, weights are calculated based on the similarity between input text and domain adapter representations. To further enhance cross-domain generalization, a dynamic LoRA fusion module is employed to integrate domain-specific knowledge from mixture of LorAs with generalized information from the mixture of data. This approach allows Sci-LoRA to effectively generalize across domains while maintaining domain-specific accuracy. Sci-LoRA significantly broadens access to cross-domain scientific content for non-expert audiences and facilitates interdisciplinary research collaboration. Contributions are summarized as:

• We propose Sci-LoRA, a model that leverages a mixture of LorAs fine-tuned on multiple scientific domains, designed for the automatic cross-domain lay paraphrasing task.

• We design the adapter weight generator and dynamic LoRA fusion that generate adaptive weights and integrate domain-specific knowledge with generalized information.

• Extensive experiments over ten evaluation metrics on five public datasets across twelve different domains demonstrate the superior effectiveness and generalization capability of Sci-LoRA over state-of-the-art models.

2 Related Work

2.1 Lay Paraphrasing

Lay paraphrasing focuses on rewriting the text written from the technical experts to the general audience without specialized domain knowledge (Cheng et al., 2025; Guo et al., 2024). Recent methods on lay summarization or lay paraphrasing tasks are restricted to a single domain. For example, large language models are finetuned and deployed to generate lay/plain language text in the biomedicine domain (Guo et al., 2024; Fonseca and Cohen, 2024; Attal et al., 2023; Tang et al., 2023; Guo et al., 2021), science and engineering domain (Cardenas et al., 2023), news domain (Liu et al., 2024c), and literature domain (Liu et al., 2023). These studies focus on developing one model within a specified domain while overlooking the generalization of the model for multiple domains. Besides, these works mainly focus on summarization instead of paraphr paraphrasing. Given the imbalanced data distribution across various domains (Cheng et al., 2025) and the growing prominence of interdisciplinary research fields, we aim to develop an effective and efficient method that performs well and demonstrates robustness for the cross-domain lay paraphrasing task.

2.2 Mixture of LorAs

As the parameter scale increases for large language models (LLMs), parameter-efficient fine-tuning (PEFT) (Houlsby et al., 2019; Han et al., 2024) strategies like Low-Rank Adaptation (LoRA) (Hu et al., 2022) efficiently adapting LLMs to multi-
ple data domains by fine-tuning a small subset of parameters. Motivated by the multisource domain adaptation of Mixture of Experts (MoE) (Cai et al., 2024; Guo et al., 2018), which is an ensemble method with a combination of sub-modules or experts for improving the overall performance in diverse tasks, mixture of LoRAs combines multiple low-rank modules to enhance adaptability and performance for more efficient and effective model tuning in different data domains.

Recent works on mixture of LoRAs can be roughly divided into three categories: (1) linear merge, where different LoRAs have the same static weights (Yadav et al., 2024; Yu et al., 2024; Huang et al., 2023); (2) router, additional linear layers for LoRA selection or rule-based LoRA selection (Feng et al., 2024; Zhang and Li, 2024; Zhao et al., 2024a; Liu et al., 2024a; Muqeth et al., 2024; Buehler and Buehler, 2024); and (3) trainable gating networks, modeling the optimal distribution of combination weights for various LoRAs (Wu et al., 2024; Xu et al., 2024; Prabhakar et al., 2024; Zhao et al., 2024c; Luo et al., 2024; Zhao et al., 2024b; Lv et al., 2024). For the lay paraphrasing or summarization task, studies leverage LoRA for efficient fine-tuning on biomedical articles (Malik et al., 2024; Kim et al., 2024). However, these works only focus on LoRA tuning in a specific biomedical domain. Motivated by mixture of LoRAs, we focus on generalization for cross-domain lay paraphrasing based on mixture of scientific LoRAs.

3 Methodology

3.1 Background

3.1.1 Low-Rank Adaptation

Low-Rank Adaptation (LoRA) (Hu et al., 2022) is a parameter-efficient fine-tuning method designed for finetuning LLMs by integrating trainable low-rank matrices instead of updating all parameters of LLMs during training. Consider a weight matrix \( W \in \mathbb{R}^{d_{in} \times d_{out}} \) within the original LLMs, where \( d_{in} \) and \( d_{out} \) are input and output dimensions, respectively. LoRA injects two low-rank matrices, \( A \in \mathbb{R}^{d_{in} \times r} \) and \( B \in \mathbb{R}^{r \times d_{out}} \), where the rank \( r \ll \min(d_{in}, d_{out}) \). Instead of directly updating \( W \), LoRA modifies model’s forward process for one layer as the following:

\[
f(x) = Wx + \Delta W x = Wx + BAx \tag{1}
\]

where \( x \in \mathbb{R}^{d_{in}} \) denotes the input.

3.1.2 Problem Formulation

Lay paraphrasing is the process of rephrasing scientific or technical language into simpler, and more accessible language that can be easily understood by a general audience. Let \( D_{multi} = \{D_1, D_2, \ldots, D_n\} \) represent \( n \) different technical domains data, where \( D_i = \{X_i, Y_i\} \). \( X_i \) is the technical text (inputs) and \( Y_i \) is the general-audience text (outputs). \( X_i = \{x_1, x_2, \cdots, x_m\} \), \( Y_i = \{y_1, y_2, \cdots, y_m\} \), where \( m \) is the number of textual documents. Given a LLM \( \mathcal{M} \), and a set of LoRA adapters \( \{\triangle \theta_1, \triangle \theta_2, \cdots, \triangle \theta_n\} \) for \( n \) different domains, where each adapter is trained on its corresponding training splitted domain data \( D_i \). In the multi-domain lay paraphrasing scenario, \( \forall x \in D_{multi} \), the generated text is expressed as:

\[
y = \mathcal{M}(\theta, \alpha_1 \cdot \triangle \theta_1, \cdots, \alpha_n \cdot \triangle \theta_n, x) \tag{2}
\]

where \( \theta \) is the original parameters of the LLM \( \mathcal{M} \), and \( \alpha \) is the weight generated from the LoRA adapters weight generator.

3.2 Sci-LoRA Framework

In this section, we describe our proposed Sci-LoRA as shown in Figure 1 for serving the mixture of LoRAs in multi-domain scenarios. Our proposed model Sci-LoRA contains three major components: the domain LoRAs training module (Sec. 3.2.1), the adapter weight generator (Sec. 3.2.2), and the dynamic LoRA fusion strategy (Sec. 3.2.3).

3.2.1 Domain LoRAs Training

We use the pre-trained Qwen2.5-7B-Instruct as the base model for Sci-LoRA because it is an open-source model under the Apache-2.0 license and it shows good performance for generating long texts (Team, 2024). For each specific domain, we train one LoRA adapter on the domain training set based on Qwen2.5-7B-Instruct model. In total, we train twelve LoRA adapters corresponding to twelve different domains. During the inference stage, Sci-LoRA can dynamically select the weighted mixture of LoRAs for a given text without domain tags.

3.2.2 Adapter Weight Generator

Although the input text originates from a specific domain, it often incorporates interdisciplinary technical knowledge. To leverage the strengths of different domain-specific LoRAs, we design an Adapter Weight Generator that effectively and dynamically
Figure 1: The Sci-LoRA framework including the adapter weight generator and dynamic LoRA fusion. All text encoder modules are the same text encoder trained on a subset of training data across all domains.

integrates and utilizes LoRAs across multiple domains. The Adapter Weight Generator includes a trained text encoder, and a weight generator.

Text Encoder  Directly using a pre-trained text encoder to obtain embeddings for the input text results in highly similar cosine similarity values with embeddings from various domain representations (Sec. 4.2.2 and Sec. C). To better differentiate representations among domains, we fine-tune a text encoder, Sentence-BERT (Reimers and Gurevych, 2019)), using contrastive learning. The text encoder is fine-tuned on a subset of training data across all domains. Specifically, the training dataset \( D \) consists of positive paired samples \((x_i, x_j)\) where \( x_j : j \neq i, x_j \in Dom(x_i) \) from the same domain, and negative paired samples \((x_i, x_k)\) where \( x_k : x_k \notin Dom(x_i) \) randomly selected from other domains. The training process is achieved through a contrastive loss (Oord et al., 2018) as:

\[
\mathcal{L} = \frac{e^{-\|x_i - x_j\|^2 / \tau}}{e^{-\|x_i - x_j\|^2 / \tau} + \sum_{k=1}^m e^{-\|x_i - x_k\|^2 / \tau}}
\]

where \( m \) is the number of negative pairs to each positive pair, and \( \tau \) is the softmax temperature.

Weight Generator  Since each LoRA is fine-tuned on domain-specific textual data, it can be represented by a set of domain-related data points. Rather than using the average embedding of randomly sampled data points from the training set, which can overlook important patterns in the data, we employ the k-means clustering algorithm to identify \( k \) clusters within the embeddings of the training set. Random sampling may select over or under-representation of certain parts of the dataset while clusting-based approach ensures a more balanced and systematic coverage. To ensure a more representative and structured sampling approach, we select the data points closest to the centroids of these clusters:

\[
e_j^* = \arg \min_{E(x_i) \in C_j} \|E(x_i) - c_j\|_2
\]

where \( c_j = \frac{1}{|C_j|} \sum_{E(x_i) \in C_j} E(x_i) \) is the centroid for its corresponding cluster \( C_j \), and \( E(\cdot) \) is the fine-tuned Text Encoder. The final representation for each domain-specific adapter is expressed as:

\[
r_{\triangle \theta_i} = \frac{1}{K} \sum_{j=1}^K e_j^*
\]

where \( K \) is the total number of clusters.

During inference, the weight \( \alpha_i \) of input text \( x_i \) for its domain adapter \( \triangle \theta_i \) can be generated as:

\[
\alpha_i = \frac{1}{1 + \|E(x_i) - r_{\triangle \theta_i}\|_2}
\]

where \( E(\cdot) \) is the fine-tuned text encoder and \( r_{\triangle \theta_i} \) is from Eq. 5.
Table 2: Dataset statistics reported in the format of technical (source) / non-technical (target) summaries.

<table>
  <tr>
    <th>Dataset</th>
    <th>#Doc.</th>
    <th>Avg. # Sentence</th>
    <th>Avg. Sentence Len.</th>
    <th>FKGL</th>
    <th>DCRS</th>
  </tr>
  <tr>
    <td>PLOS</td>
    <td>27,525</td>
    <td>11.99 / 8.32</td>
    <td>23.30 / 23.51</td>
    <td>15.04 / 14.76</td>
    <td>11.06 / 10.91</td>
  </tr>
  <tr>
    <td>eLife</td>
    <td>4,828</td>
    <td>7.18 / 17.97</td>
    <td>23.14 / 20.56</td>
    <td>15.57 / 10.92</td>
    <td>11.78 / 8.83</td>
  </tr>
  <tr>
    <td>CELLS</td>
    <td>62,886</td>
    <td>12.82 / 7.54</td>
    <td>22.08 / 22.63</td>
    <td>16.79 / 16.67</td>
    <td>6.78 / 7.01</td>
  </tr>
  <tr>
    <td>SciTechNews</td>
    <td>2,431</td>
    <td>8.78 / 6.59</td>
    <td>22.31 / 23.87</td>
    <td>16.33 / 15.80</td>
    <td>7.14 / 7.59</td>
  </tr>
  <tr>
    <td>VTechAGP</td>
    <td>4,409</td>
    <td>13.68 / 11.66</td>
    <td>27.68 / 26.38</td>
    <td>15.74 / 14.78</td>
    <td>6.92 / 6.72</td>
  </tr>
</table>

3.2.3 Dynamic LoRA Fusion
To effectively balance domain-specific knowledge and generalization, we propose dynamic LoRA fusion module consisting of domain-specific LoRA aggregation and unified LoRA for multi-domain generalization. First, all domain-specific LoRAs are merged with learned LoRA weights \( \alpha \) in Eq. 6. We proceed to integrate these LoRA parameters \( \triangle \theta \) into the LLM with parameter \( \theta \). The specialized representation for text input \( x_i \) is expressed as:

\[
r_{specialized} = \mathcal{M}(\theta + \sum_{i=1}^{n} a_i \triangle \theta_i, x_i)
\]

where \( n \) is the number of domains. Next, a single LoRA \( \triangle \theta_0 \) is trained on data across all domains to capture more generalized features. The generalized representation for input text \( x_i \) is expressed as:

\[
r_{generalized} = \mathcal{M}(\theta + \triangle \theta_0, x_i)
\]

where \( \theta \) is the original parameter for model \( \mathcal{M} \). This ensures that broad patterns across domains are captured, avoiding overfitting to specific domain. Finally, we combine \( r_{specialized} \) and \( r_{generalized} \) as:

\[
\hat{r} = \beta \cdot r_{specialized} + (1 - \beta) \cdot r_{generalized}
\]

where \( \beta \) controls the balance between domain-specialized and general knowledge.

4 Experiments

4.1 Evaluation Settings

4.1.1 Datasets
In our experiments, we evaluate our proposed model over five public datasets: PLOS (Goldsack et al., 2022), eLife (Goldsack et al., 2022), CELLS (Guo et al., 2024), SciTechNews (Cardenas et al., 2023), and VTechAGP (Cheng et al., 2025). All these public datasets include technical abstracts and corresponding non-technical summaries. We follow the same training, validation, and testing splits as these datasets originally have used 1. Table 2 provides detailed statistics of each dataset. Description of each dataset and explanation of Table 2 are provided in Sec. B in the Appendix. Given the multiple domains and the unbalanced training data shown in Table 2, we conduct experiments to explore the generalization of Sci-LoRA and other LLMs for the cross-domain lay paraphrasing task.

4.1.2 Baselines and Evaluation Metrics
We compare Sci-LoRA with the following state-of-the-art LLM baselines: ChatGPT (gpt-3.5-turbo-0613) (Brown et al., 2020), GPT-4o (Achiam et al., 2023), Phi-3 (Phi-3-mini-128k-instruct) (Abdin et al., 2024), Phi-3.5 (Phi-3.5-mini-instruct), OPT (opt-6.7b) (Zhang et al., 2022), LLaMA3 (Llama-3.2-3B-Instruct) (Dubey et al., 2024), Qwen2.5 (Qwen2.5-7B-Instruct) (Team, 2024), Mistral (Mistral-7B-Instruct-v0.3) (Jiang et al., 2023), Mixtral (Mixtral-8x7B-Instruct-v0.1) (Jiang et al., 2024), and DSPTS (Cheng et al., 2025). For evaluation metrics, we follow (Cheng et al., 2025) to use embedding-based metrics: BERTScore (F1) (Zhang* et al., 2020), BLONDE (F1) (Jiang et al., 2022), sentence-level and document-level BLEU (Lin and Och, 2004); Word-based metrics: ROUGE1, ROUGE2 (Lin, 2004) and METEOR (Banerjee and Lavie, 2005); End-to-end metrics: COMET (Rei et al., 2022); Simplicity: SARI (Xu et al., 2016), and Readability: FRES (Flesch Reading-Ease Score) (Flesch, 1979).

4.1.3 Parameter Settings
We implement Sci-LoRA in PyTorch and fine-tune various LoRA adapters for each domain using LLaMA-Factory (Zheng et al., 2024). For domain LoRAs training, the learning rate is 1e-4, batch size is 4 per device, LoRA rank is 8, document maximum length is 2048. For adapter weight generator, the learning rate is 1e-5, batch size is 16, sampling size is 500, the number of clusters is 10. It is cho-

1We re-split SciTechNews as the original training set does not have the abstract and summary pairs.
Table 3: Results over six evaluation metrics. Another four metrics are reported in Table 4.

<table>
  <tr>
    <th></th>
    <th>CELLS</th>
    <th>PLOS</th>
    <th>eLife</th>
    <th>News</th>
    <th>ALS</th>
    <th>AAD</th>
    <th>ENG</th>
    <th>LAHS</th>
    <th>NRE</th>
    <th>SCI</th>
    <th>BUS</th>
    <th>VM</th>
    <th>d-BLEUE (%)</th>
  </tr>
  <tr>
    <td>OPT</td><td>4.45</td><td>5.31</td><td>2.45</td><td>2.50</td><td>24.88</td><td>23.10</td><td>23.00</td><td>36.78</td><td>20.26</td><td>19.44</td><td>20.16</td><td>14.32</td><td></td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>4.38</td><td>3.51</td><td>3.27</td><td>2.61</td><td>20.73</td><td>20.78</td><td>19.61</td><td>21.92</td><td>20.77</td><td>19.35</td><td>22.40</td><td>18.61</td><td></td>
  </tr>
  <tr>
    <td>Phi-4</td><td>6.31</td><td>6.30</td><td>1.90</td><td>1.32</td><td>25.55</td><td>28.71</td><td>24.45</td><td>36.59</td><td>24.42</td><td>20.96</td><td>29.67</td><td>21.78</td><td></td>
  </tr>
  <tr>
    <td>Mistral</td><td>5.27</td><td>4.47</td><td>3.12</td><td>2.61</td><td>28.62</td><td>28.63</td><td>24.53</td><td>22.14</td><td>11.55</td><td>17.18</td><td>16.32</td><td>13.66</td><td></td>
  </tr>
  <tr>
    <td>Mixtral</td><td>6.36</td><td>7.43</td><td>3.05</td><td>2.88</td><td>13.64</td><td>31.22</td><td>14.96</td><td>24.00</td><td>13.58</td><td>14.93</td><td>13.12</td><td>7.29</td><td></td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td>9.26</td><td>10.18</td><td>3.98</td><td>4.26</td><td>25.55</td><td>28.71</td><td>24.40</td><td>36.59</td><td>24.42</td><td>20.56</td><td>23.55</td><td>24.33</td><td></td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>6.70</td><td>7.08</td><td>1.74</td><td>3.07</td><td>15.39</td><td>15.59</td><td>13.75</td><td>16.99</td><td>14.11</td><td>12.84</td><td>17.20</td><td>13.62</td><td></td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>5.10</td><td>5.76</td><td>2.65</td><td>2.92</td><td>11.05</td><td>12.40</td><td>9.65</td><td>13.60</td><td>10.78</td><td>9.05</td><td>9.56</td><td>9.10</td><td></td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>24.95</td><td>33.53</td><td>24.98</td><td>38.80</td><td>28.11</td><td>21.31</td><td>33.41</td><td>23.42</td><td></td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td><b>11.15</b></td><td><b>12.43</b></td><td><b>6.09</b></td><td><b>4.61</b></td><td><b>31.03</b></td><td><b>38.97</b></td><td><b>28.31</b></td><td><b>40.33</b></td><td><b>29.61</b></td><td><b>23.31</b></td><td><b>32.86</b></td><td><b>29.55</b></td><td></td>
  </tr>
</table>

<table>
  <tr>
    <th></th>
    <th colspan="12">BERTScore (F1 %)</th>
  </tr>
  <tr>
    <td>OPT</td><td>81.53</td><td>82.47</td><td>77.53</td><td>76.08</td><td>82.09</td><td>81.01</td><td>79.70</td><td>82.41</td><td>83.46</td><td>79.58</td><td>84.60</td><td>82.30</td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>80.22</td><td>75.11</td><td>74.86</td><td>77.12</td><td>82.59</td><td>80.72</td><td>81.57</td><td>82.72</td><td>82.60</td><td>81.44</td><td>83.20</td><td>81.64</td>
  </tr>
  <tr>
    <td>Phi-4</td><td>82.64</td><td>83.04</td><td>78.28</td><td>76.20</td><td>84.98</td><td>82.55</td><td>83.57</td><td>86.24</td><td>84.75</td><td>82.91</td><td>85.22</td><td>84.37</td>
  </tr>
  <tr>
    <td>Mistral</td><td>79.36</td><td>80.29</td><td>80.48</td><td>77.01</td><td>84.69</td><td>81.86</td><td>83.31</td><td>82.82</td><td>81.27</td><td>80.13</td><td>80.73</td><td>81.92</td>
  </tr>
  <tr>
    <td>Mixtral</td><td>81.38</td><td>82.04</td><td>80.49</td><td>76.80</td><td>80.92</td><td>77.09</td><td>79.26</td><td>80.79</td><td>78.50</td><td>78.39</td><td>78.66</td><td>70.90</td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td>82.36</td><td>82.70</td><td>77.40</td><td>79.48</td><td>82.55</td><td>81.31</td><td>86.24</td><td>84.75</td><td>82.85</td><td>84.11</td><td>84.47</td><td>84.37</td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>82.14</td><td>82.62</td><td>80.35</td><td>77.83</td><td>84.67</td><td>82.48</td><td>83.52</td><td>84.75</td><td>84.65</td><td>83.05</td><td>85.17</td><td>83.89</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>81.13</td><td>81.81</td><td>80.23</td><td>77.34</td><td>83.55</td><td>81.80</td><td>81.81</td><td>83.88</td><td>83.43</td><td>81.32</td><td>48.51</td><td>82.53</td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>85.48</td><td>83.70</td><td><b>84.41</b></td><td><b>87.27</b></td><td>85.51</td><td>82.90</td><td><b>87.97</b></td><td>84.20</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td><b>83.00</b></td><td><b>83.35</b></td><td><b>81.40</b></td><td><b>78.82</b></td><td><b>86.01</b></td><td>84.37</td><td>84.30</td><td>87.25</td><td><b>86.18</b></td><td><b>83.51</b></td><td>86.51</td><td><b>85.90</b></td>
  </tr>
</table>

<table>
  <tr>
    <th></th>
    <th colspan="2">BLONDE (F1 %)</th>
  </tr>
  <tr>
    <td>OPT</td><td>14.08</td><td>14.33</td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>14.02</td><td>14.54</td>
  </tr>
  <tr>
    <td>Phi-4</td><td>17.97</td><td>5.07</td>
  </tr>
  <tr>
    <td>Mistral</td><td>16.28</td><td>10.87</td>
  </tr>
  <tr>
    <td>Mixtral</td><td>17.55</td><td>9.37</td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td><b>18.80</b></td><td><b>24.34</b></td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>18.78</td><td>7.14</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>16.19</td><td>17.81</td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td>18.12</td><td>16.81</td>
  </tr>
</table>

<table>
  <tr>
    <th></th>
    <th colspan="9"> ROUGE1 (%)</th>
  </tr>
  <tr>
    <td>OPT</td><td>35.43</td><td>33.49</td><td>30.43</td><td>28.51</td><td>48.22</td><td>29.95</td><td>42.26</td><td>48.68</td><td>48.02</td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>33.90</td><td>37.81</td><td>31.24</td><td>30.40</td><td>45.65</td><td>41.42</td><td>48.86</td><td>45.01</td><td>46.06</td>
  </tr>
  <tr>
    <td>Phi-4</td><td>39.46</td><td>41.51</td><td>28.11</td><td>25.17</td><td>52.90</td><td>46.51</td><td>48.39</td><td>50.63</td><td>51.67</td>
  </tr>
  <tr>
    <td>Mistral</td><td>34.44</td><td>33.42</td><td>35.63</td><td>28.09</td><td>52.35</td><td>45.53</td><td>48.20</td><td>52.32</td><td>47.79</td>
  </tr>
  <tr>
    <td>Mixtral</td><td>34.77</td><td>40.84</td><td>36.75</td><td>30.00</td><td>43.44</td><td>37.31</td><td>45.93</td><td>42.41</td><td>45.20</td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td>40.55</td><td>42.75</td><td>37.57</td><td>31.85</td><td>52.90</td><td>46.51</td><td>47.54</td><td>50.03</td><td>48.02</td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>38.42</td><td>40.55</td><td>31.74</td><td>30.25</td><td>51.50</td><td>45.61</td><td>47.97</td><td>52.03</td><td>51.88</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>37.27</td><td>39.36</td><td>30.65</td><td>30.20</td><td>47.52</td><td>42.74</td><td>43.21</td><td>47.05</td><td>48.17</td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>54.88</td><td>51.18</td><td><b>51.73</b></td><td><b>59.95</b></td><td><b>54.78</b></td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td><b>43.10</b></td><td><b>45.59</b></td><td><b>42.59</b></td><td><b>32.68</b></td><td><b>56.90</b></td><td><b>52.69</b></td><td>54.55</td><td><b>57.30</b></td><td>KL96: 58.50</td>
  </tr>
</table>

<table>
  <tr>
    <th></th>
    <th colspan="9">METEOR (%)</th>
  </tr>
  <tr>
    <td>OPT</td><td>27.70</td><td>25.45</td><td>21.76</td><td>8.06</td><td>37.98</td><td>28.90</td><td>34.35</td><td>44.82</td><td>33.88</td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>29.29</td><td>23.55</td><td>20.18</td><td>87.58</td><td>41.00</td><td>33.67</td><td>32.02</td><td>45.51</td><td>33.49</td>
  </tr>
  <tr>
    <td>Phi-4</td><td><b>36.83</b></td><td><b>36.75</b></td><td>17.74</td><td>22.03</td><td>40.88</td><td>39.85</td><td>38.08</td><td>49.26</td><td>41.37</td>
  </tr>
  <tr>
    <td>Mistral</td><td>27.98</td><td>24.97</td><td>20.70</td><td>21.61</td><td>41.43</td><td>45.38</td><td>40.94</td><td>44.38</td><td>38.62</td>
  </tr>
  <tr>
    <td>Mixtral</td><td>30.92</td><td>31.91</td><td>19.56</td><td>22.79</td><td>32.47</td><td>30.54</td><td>29.97</td><td>38.30</td><td>27.88</td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td>30.40</td><td>31.88</td><td>21.21</td><td>22.90</td><td>40.88</td><td>39.38</td><td>38.74</td><td>49.26</td><td>41.31</td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>38.55</td><td>28.15</td><td>16.07</td><td>21.21</td><td>38.91</td><td>36.68</td><td>31.73</td><td>31.76</td><td>35.33</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>28.11</td><td>31.18</td><td>18.88</td><td>24.01</td><td>33.77</td><td>33.32</td><td>31.73</td><td>36.68</td><td>37.77</td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td><td>-</td><td>-</td><td>20.54</td><td>41.18</td><td>52.54</td><td>45.02</td><td>45.03</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td>32.29</td><td>35.02</td><td><b>28.98</b></td><td>23.87</td><td><b>45.51</b></td><td><b>42.23</b></td><td>53.70</td><td><b>47.59</b></td><td><b>48.82</b></td>
  </tr>
</table>

<table>
  <tr>
    <th></th>
    <th colspan="9">SARI</th>
  </tr>
  <tr>
    <td>OPT</td><td>37.27</td><td>37.54</td><td>37.27</td><td>39.69</td><td>35.38</td><td>31.92</td><td>37.24</td><td>36.49</td><td>34.66</td>
  </tr>
  <tr>
    <td>LLaMA3</td><td>40.13</td><td>37.61</td><td>42.47</td><td>40.70</td><td>35.93</td><td>34.45</td><td>35.61</td><td>33.17</td><td>35.10</td>
  </tr>
  <tr>
    <td>Phi-4</td><td>40.95</td><td>42.26</td><td>40.46</td><td>37.83</td><td>40.14</td><td>39.39</td><td>38.73</td><td>38.98</td><td>48.19</td>
  </tr>
  <tr>
    <td>Mistral</td><td><b>41.69</b></td><td>38.72</td><td>45.23</td><td>43.68</td><td>40.86</td><td>43.41</td><td>41.32</td><td>32.40</td><td>33.09</td>
  </tr>
  <tr>
    <td>Mixtral</td><td>40.39</td><td>39.95</td><td>37.45</td><td>43.93</td><td>25.38</td><td>35.53</td><td>34.37</td><td>30.77</td><td>31.46</td>
  </tr>
  <tr>
    <td>Qwen2.5</td><td>40.38</td><td>43.08</td><td>44.03</td><td>42.01</td><td>44.19</td><td>38.77</td><td>39.93</td><td>38.79</td><td>38.97</td>
  </tr>
  <tr>
    <td>GPT-3.5</td><td>40.12</td><td>39.60</td><td>46.22</td><td>35.94</td><td>37.83</td><td>35.41</td><td>36.90</td><td>35.26</td><td>35.59</td>
  </tr>
  <tr>
    <td>GPT-4o</td><td>39.93</td><td>39.60</td><td>43.04</td><td>39.48</td><td>35.95</td><td>36.65</td><td>34.96</td><td>34.59</td><td>36.53</td>
  </tr>
  <tr>
    <td>DSPTS</td><td>-</td><td>-</td><td>-</td><td>37.31</td><td>6.01</td><td>38.21</td><td>38.96</td><td>37.33</td><td>31.11</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td><td>41.13</td><td>40.07</td><td><b>47.64</b></td><td><b>43.76</b></td><td><b>44.32</b></td><td><b>44.64</b></td><td><b>42.88</b></td><td><b>41.77</b></td><td>40.60</td>
  </tr>
</table>
Table 4: Results of the remaining four evaluation metrics over all baselines. The abbreviations stands for CELLS, PLOS, eLife, SciTechNews, Agriculture and Life Science, Architecture, Arts and Design, Engineering, Liberal Arts and Human Sciences, Natural Resources and Environment, Science, Business, and Veterinary Medicine.

<table>
  <tr>
    <th></th>
    <th>CELLS</th>
    <th>PLOS</th>
    <th>eLife</th>
    <th>News</th>
    <th>ALS</th>
    <th>AAD</th>
    <th>ENG</th>
    <th>LAHS</th>
    <th>NRE</th>
    <th>SCI</th>
    <th>BUS</th>
    <th>VM</th>
  </tr>
  <tr>
    <th></th>
    <th>s-BLEU (%)</th>
    <th>ROUGE2 (%)</th>
    <th>COMET (%)</th>
    <th>FRES</th>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>1.84</td>
    <td>1.35</td>
    <td>0.58</td>
    <td>0.84</td>
    <td>8.37</td>
    <td>14.42</td>
    <td>12.72</td>
    <td>15.36</td>
    <td>7.02</td>
    <td>5.20</td>
    <td>12.61</td>
    <td>8.20</td>
  </tr>
  <tr>
    <td>Phi-4</td>
    <td><b>4.08</b></td>
    <td><b>4.58</b></td>
    <td>0.44</td>
    <td>1.33</td>
    <td>8.95</td>
    <td>13.43</td>
    <td>9.72</td>
    <td>19.82</td>
    <td>7.55</td>
    <td>8.22</td>
    <td>16.03</td>
    <td>9.32</td>
  </tr>
  <tr>
    <td>Mistral</td>
    <td>1.51</td>
    <td>1.42</td>
    <td>0.85</td>
    <td>0.76</td>
    <td><b>11.46</b></td>
    <td>14.70</td>
    <td><b>13.74</b></td>
    <td>11.72</td>
    <td>5.51</td>
    <td>3.89</td>
    <td>12.10</td>
    <td>10.37</td>
  </tr>
  <tr>
    <td>Mixtral</td>
    <td>2.55</td>
    <td>2.58</td>
    <td>0.82</td>
    <td>0.92</td>
    <td>4.03</td>
    <td>4.52</td>
    <td>3.94</td>
    <td>6.52</td>
    <td>4.40</td>
    <td>4.05</td>
    <td>1.75</td>
    <td>4.05</td>
  </tr>
  <tr>
    <td>Qwen2.5</td>
    <td>3.07</td>
    <td>3.23</td>
    <td><b>1.22</b></td>
    <td>1.58</td>
    <td>8.95</td>
    <td>13.43</td>
    <td>9.72</td>
    <td>19.82</td>
    <td>7.55</td>
    <td>8.22</td>
    <td>15.49</td>
    <td>9.32</td>
  </tr>
  <tr>
    <td>GPT-3.5</td>
    <td>2.56</td>
    <td>2.58</td>
    <td>0.75</td>
    <td>1.10</td>
    <td>4.53</td>
    <td>5.74</td>
    <td>4.34</td>
    <td>6.25</td>
    <td>4.08</td>
    <td>4.20</td>
    <td>8.17</td>
    <td>4.51</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>2.15</td>
    <td>2.12</td>
    <td>0.75</td>
    <td>0.88</td>
    <td>3.84</td>
    <td>3.61</td>
    <td>3.51</td>
    <td>6.70</td>
    <td>4.06</td>
    <td>3.32</td>
    <td>3.42</td>
    <td>3.55</td>
  </tr>
  <tr>
    <td>DSPT5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>10.84</td>
    <td>14.09</td>
    <td>11.25</td>
    <td>22.12</td>
    <td>9.19</td>
    <td>8.13</td>
    <td><b>20.30</b></td>
    <td><b>11.38</b></td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>3.99</b></td>
    <td><b>4.06</b></td>
    <td><b>1.22</b></td>
    <td><b>2.20</b></td>
    <td><b>10.21</b></td>
    <td><b>18.38</b></td>
    <td><b>11.69</b></td>
    <td><b>23.99</b></td>
    <td><b>13.84</b></td>
    <td><b>9.86</b></td>
    <td><b>15.76</b></td>
    <td><b>11.32</b></td>
  </tr>
  <tr>
    <td>OPT</td>
    <td>9.77</td>
    <td>10.91</td>
    <td>5.77</td>
    <td>5.79</td>
    <td>26.78</td>
    <td>21.90</td>
    <td>22.67</td>
    <td>33.34</td>
    <td>21.11</td>
    <td>20.79</td>
    <td>21.38</td>
    <td>14.29</td>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>7.97</td>
    <td>6.11</td>
    <td>6.51</td>
    <td>5.28</td>
    <td>24.62</td>
    <td>23.17</td>
    <td>22.99</td>
    <td>36.49</td>
    <td>24.84</td>
    <td>22.53</td>
    <td>15.58</td>
    <td>12.09</td>
  </tr>
  <tr>
    <td>Phi-4</td>
    <td><b>14.59</b></td>
    <td>15.80</td>
    <td>5.86</td>
    <td>4.72</td>
    <td>27.50</td>
    <td>23.37</td>
    <td>23.38</td>
    <td>35.59</td>
    <td>26.53</td>
    <td>21.45</td>
    <td>29.32</td>
    <td>22.02</td>
  </tr>
  <tr>
    <td>Mistral</td>
    <td>8.98</td>
    <td>6.75</td>
    <td>7.76</td>
    <td>4.07</td>
    <td>28.99</td>
    <td>28.18</td>
    <td>26.58</td>
    <td>30.50</td>
    <td>23.37</td>
    <td>18.78</td>
    <td>18.36</td>
    <td>12.98</td>
  </tr>
  <tr>
    <td>Mixtral</td>
    <td>9.83</td>
    <td>11.26</td>
    <td>9.09</td>
    <td>4.92</td>
    <td>16.25</td>
    <td>18.98</td>
    <td>13.83</td>
    <td>18.92</td>
    <td>15.82</td>
    <td>13.81</td>
    <td>14.14</td>
    <td>11.44</td>
  </tr>
  <tr>
    <td>Qwen2.5</td>
    <td>11.92</td>
    <td>13.25</td>
    <td>9.23</td>
    <td>6.23</td>
    <td>27.50</td>
    <td>23.37</td>
    <td>23.20</td>
    <td>35.59</td>
    <td>26.53</td>
    <td>20.59</td>
    <td>23.35</td>
    <td>22.02</td>
  </tr>
  <tr>
    <td>GPT-3.5</td>
    <td>9.54</td>
    <td>10.50</td>
    <td>7.63</td>
    <td>5.06</td>
    <td>20.08</td>
    <td>16.75</td>
    <td>17.88</td>
    <td>21.89</td>
    <td>19.67</td>
    <td>16.55</td>
    <td>21.76</td>
    <td>17.54</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>8.70</td>
    <td>9.81</td>
    <td>8.04</td>
    <td>4.96</td>
    <td>14.87</td>
    <td>13.54</td>
    <td>12.43</td>
    <td>17.23</td>
    <td>14.75</td>
    <td>11.36</td>
    <td>14.58</td>
    <td>12.15</td>
  </tr>
  <tr>
    <td>DSPT5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>28.58</td>
    <td>27.55</td>
    <td>27.02</td>
    <td>40.24</td>
    <td>30.83</td>
    <td>21.97</td>
    <td><b>39.61</b></td>
    <td>24.63</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>14.02</td>
    <td><b>15.89</b></td>
    <td><b>11.31</b></td>
    <td><b>6.90</b></td>
    <td><b>32.16</b></td>
    <td><b>30.98</b></td>
    <td><b>27.62</b></td>
    <td><b>40.92</b></td>
    <td><b>33.19</b></td>
    <td><b>24.26</b></td>
    <td><b>34.50</b></td>
    <td><b>28.24</b></td>
  </tr>
  <tr>
    <td>OPT</td>
    <td>74.74</td>
    <td>77.73</td>
    <td>74.74</td>
    <td>71.86</td>
    <td>80.45</td>
    <td>71.81</td>
    <td>81.42</td>
    <td>77.90</td>
    <td>78.89</td>
    <td>76.13</td>
    <td>81.01</td>
    <td>76.41</td>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>79.17</td>
    <td>77.43</td>
    <td>78.40</td>
    <td>74.80</td>
    <td>82.92</td>
    <td>76.06</td>
    <td>82.80</td>
    <td>81.87</td>
    <td>84.54</td>
    <td>80.07</td>
    <td>83.39</td>
    <td>83.44</td>
  </tr>
  <tr>
    <td>Phi-4</td>
    <td>78.87</td>
    <td>80.05</td>
    <td>73.62</td>
    <td>70.84</td>
    <td>83.24</td>
    <td>76.06</td>
    <td>82.02</td>
    <td>81.86</td>
    <td>83.23</td>
    <td>79.90</td>
    <td>83.56</td>
    <td>83.58</td>
  </tr>
  <tr>
    <td>Mistral</td>
    <td>77.84</td>
    <td>78.45</td>
    <td>80.13</td>
    <td>74.46</td>
    <td>81.05</td>
    <td>72.00</td>
    <td>79.22</td>
    <td>79.72</td>
    <td>81.01</td>
    <td>79.23</td>
    <td>81.31</td>
    <td>82.42</td>
  </tr>
  <tr>
    <td>Mixtral</td>
    <td>79.46</td>
    <td>79.26</td>
    <td>77.98</td>
    <td>74.24</td>
    <td>78.57</td>
    <td>68.67</td>
    <td>75.52</td>
    <td>75.98</td>
    <td>73.99</td>
    <td>74.46</td>
    <td>69.64</td>
    <td>68.16</td>
  </tr>
  <tr>
    <td>Qwen2.5</td>
    <td>78.95</td>
    <td>79.90</td>
    <td>79.09</td>
    <td>74.32</td>
    <td>83.42</td>
    <td>76.06</td>
    <td>81.22</td>
    <td>81.86</td>
    <td>83.23</td>
    <td>80.33</td>
    <td>82.60</td>
    <td>83.58</td>
  </tr>
  <tr>
    <td>GPT-3.5</td>
    <td><b>80.23</b></td>
    <td>79.69</td>
    <td>77.20</td>
    <td>75.22</td>
    <td><b>85.49</b></td>
    <td><b>80.30</b></td>
    <td><b>84.00</b></td>
    <td><b>83.29</b></td>
    <td><b>85.79</b></td>
    <td><b>82.98</b></td>
    <td>85.29</td>
    <td>84.10</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>79.97</td>
    <td>80.26</td>
    <td>77.50</td>
    <td>75.39</td>
    <td>84.47</td>
    <td>78.71</td>
    <td>81.72</td>
    <td>82.33</td>
    <td>84.95</td>
    <td>80.67</td>
    <td>83.86</td>
    <td>83.71</td>
  </tr>
  <tr>
    <td>DSPT5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>81.37</td>
    <td>75.90</td>
    <td>80.64</td>
    <td>81.76</td>
    <td>82.09</td>
    <td>77.15</td>
    <td>84.07</td>
    <td>78.36</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>79.10</td>
    <td><b>80.29</b></td>
    <td><b>82.95</b></td>
    <td><b>76.00</b></td>
    <td>83.79</td>
    <td>78.02</td>
    <td>82.90</td>
    <td>82.62</td>
    <td>84.66</td>
    <td>80.87</td>
    <td><b>85.40</b></td>
    <td><b>84.67</b></td>
  </tr>
  <tr>
    <td>OPT</td>
    <td>34.05</td>
    <td>34.26</td>
    <td>34.05</td>
    <td>39.77</td>
    <td>34.05</td>
    <td>40.87</td>
    <td>31.62</td>
    <td>29.69</td>
    <td>42.11</td>
    <td>32.73</td>
    <td>21.33</td>
    <td>32.43</td>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>49.55</td>
    <td>50.57</td>
    <td>50.36</td>
    <td><b>49.86</b></td>
    <td>50.16</td>
    <td>49.75</td>
    <td>41.29</td>
    <td>39.67</td>
    <td>49.25</td>
    <td>40.79</td>
    <td>40.58</td>
    <td>49.86</td>
  </tr>
  <tr>
    <td>Phi-4</td>
    <td>34.15</td>
    <td>35.24</td>
    <td>23.36</td>
    <td>33.47</td>
    <td>35.07</td>
    <td>51.89</td>
    <td>32.63</td>
    <td>30.50</td>
    <td>42.82</td>
    <td>33.65</td>
    <td>21.43</td>
    <td>42.31</td>
  </tr>
  <tr>
    <td>Mistral</td>
    <td><b>51.06</b></td>
    <td><b>53.10</b></td>
    <td>53.61</td>
    <td>42.72</td>
    <td>33.75</td>
    <td>50.57</td>
    <td>31.72</td>
    <td><b>41.09</b></td>
    <td><b>51.38</b></td>
    <td><b>43.65</b></td>
    <td><b>41.70</b></td>
    <td>43.02</td>
  </tr>
  <tr>
    <td>Mixtral</td>
    <td>49.35</td>
    <td>49.15</td>
    <td>48.94</td>
    <td>41.40</td>
    <td>41.40</td>
    <td>41.29</td>
    <td>32.73</td>
    <td>39.37</td>
    <td>41.09</td>
    <td>41.19</td>
    <td>39.16</td>
    <td>40.08</td>
  </tr>
  <tr>
    <td>Qwen2.5</td>
    <td>43.43</td>
    <td>43.43</td>
    <td>53.41</td>
    <td>42.11</td>
    <td>35.07</td>
    <td><b>51.98</b></td>
    <td>32.43</td>
    <td>30.50</td>
    <td>42.82</td>
    <td>33.34</td>
    <td>23.04</td>
    <td>42.31</td>
  </tr>
  <tr>
    <td>GPT-3.5</td>
    <td>41.29</td>
    <td>41.80</td>
    <td>40.69</td>
    <td>33.14</td>
    <td>42.11</td>
    <td>38.85</td>
    <td>31.72</td>
    <td>41.70</td>
    <td>34.15</td>
    <td>29.69</td>
    <td>42.21</td>
    <td>42.21</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>41.90</td>
    <td>42.00</td>
    <td>49.04</td>
    <td>42.31</td>
    <td><b>50.97</b></td>
    <td><b>43.83</b></td>
    <td><b>42.31</b></td>
    <td>40.99</td>
    <td>50.36</td>
    <td>42.11</td>
    <td><b>32.83</b></td>
    <td><b>50.16</b></td>
  </tr>
  <tr>
    <td>DSPT5</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>33.65</td>
    <td>41.50</td>
    <td>31.62</td>
    <td>28.17</td>
    <td>41.19</td>
    <td>32.33</td>
    <td>21.74</td>
    <td>33.34</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>44.36</td>
    <td>44.36</td>
    <td><b>54.93</b></td>
    <td>41.38</td>
    <td>44.86</td>
    <td>41.80</td>
    <td>33.82</td>
    <td>32.48</td>
    <td>42.80</td>
    <td>33.44</td>
    <td>22.53</td>
    <td>34.12</td>
  </tr>
</table>

Table 5: Ablation study over Sci-LoRA components in College of Architecture, Arts, and Design domain from VTechAGP Dataset. Due to limited space, ablation results on all domains are in Table 8 and Table 9 in Appendix.

<table>
  <tr>
    <th>Model</th>
    <th>s-BLEU</th>
    <th>d-BLEU</th>
    <th>BERTScore</th>
    <th>BLONDE</th>
    <th>ROUGE1</th>
    <th>ROUGE2</th>
    <th>METEOR</th>
    <th>COMET</th>
    <th>SARI</th>
    <th>FRES</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>5.63</td>
    <td>12.69</td>
    <td>81.97</td>
    <td>27.56</td>
    <td>42.46</td>
    <td>13.87</td>
    <td>33.98</td>
    <td><b>80.05</b></td>
    <td>34.64</td>
    <td>35.17</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>19.78</td>
    <td>34.39</td>
    <td>83.30</td>
    <td>46.43</td>
    <td>47.83</td>
    <td>26.51</td>
    <td>41.77</td>
    <td>77.20</td>
    <td>40.76</td>
    <td>42.41</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>13.43</td>
    <td>28.71</td>
    <td>82.55</td>
    <td>40.64</td>
    <td>46.51</td>
    <td>23.37</td>
    <td>39.85</td>
    <td>76.06</td>
    <td>39.39</td>
    <td><b>51.89</b></td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>23.08</td>
    <td>34.74</td>
    <td>83.46</td>
    <td>48.14</td>
    <td>48.81</td>
    <td>26.16</td>
    <td>40.48</td>
    <td>77.30</td>
    <td>41.58</td>
    <td>42.41</td>
  </tr>
  <tr>
    <td>AWG<sub>Means</sub></td>
    <td><b>23.17</b></td>
    <td>37.08</td>
    <td>83.72</td>
    <td>50.02</td>
    <td>51.13</td>
    <td>28.28</td>
    <td>43.91</td>
    <td>77.83</td>
    <td>43.56</td>
    <td>42.31</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>18.06</td>
    <td>38.62</td>
    <td>84.03</td>
    <td>50.00</td>
    <td>52.07</td>
    <td>30.86</td>
    <td>44.89</td>
    <td>77.97</td>
    <td>44.01</td>
    <td>41.29</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>14.86</td>
    <td>31.22</td>
    <td>81.59</td>
    <td>44.02</td>
    <td>50.82</td>
    <td>22.62</td>
    <td>41.42</td>
    <td>78.59</td>
    <td>43.63</td>
    <td>44.03</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>18.38</b></td>
    <td><b>38.97</b></td>
    <td><b>84.37</b></td>
    <td><b>50.99</b></td>
    <td><b>52.69</b></td>
    <td><b>30.98</b></td>
    <td><b>46.71</b></td>
    <td>78.02</td>
    <td><b>44.26</b></td>
    <td><b>41.80</b></td>
  </tr>
</table>
sen by grid search {5, 10, 15, 20} on the validation set. The weight parameter \( \beta \) is 0.5. We use the same prompt "Generate another version of the provided text for general audiences" for Sci-LoRA and all baselines. For LoRA tuning Sci-LoRA, we follow the early stopping strategy when selecting the model for testing. The model is evaluated on the validation set after every training epoch. The time for training is around 3 hours. The experiments are conducted on eight Nvidia A100 GPUs.

4.2 Results and Analysis

4.2.1 Main Results
The experimental results across five datasets spanning twelve domains are presented in Table 3 and Table 4. Note that Mixtral, GPT-3.5, and GPT-4o are pre-trained models, DSPT5 is a full-size model fine-tuned separately for each domain, and all other models utilize LoRA fine-tuning applied to a fusion of all data. We observe that: (1) In general, fine-tuned models outperform pre-trained generalist models such as Mixtral and GPT-4o, highlighting the necessity of fine-tuning for specialized tasks. (2) The fully fine-tuned DSPT5 model outperforms other LoRA fine-tuned baselines because it is fine-tuned separately for each domain. Note that other LoRA refers to a single generalist LoRA finetuned on all domains. This ensures that its performance is not influenced by data from other domains, allowing it to specialize more effectively. In contrast, the LoRA fine-tuned baselines use a single LoRA adapter trained on data from all domains, which may lead to cross-domain interference and reduced specialization. (3) Our proposed Sci-LoRA achieves the best performance among all pre-trained models, LoRA fine-tuned models, and separate domain fine-tuned DSPT5. We think that Sci-LoRA effectively mitigates cross-domain interference while improving generalization by dynamically fusing multiple LoRAs with adaptive weighting.

4.2.2 Ablation Study

To assess the performance of each component in Sci-LoRA, we conducted the ablation study as shown in Table 5. Note that all models use the same base model of Qwen2.5-7B-Instruct for fair comparison. We explore different methods of adapters training, adapters weight generation (AWG) and adapters weight fusion. We observe that: (1) In general, fine-tuning separate LoRAs for each domain (Multi-LoRAs) yields better performance than using a single LoRA across all domains. This is because a single LoRA captures more generalized information, making it less effective for domain-specific tasks. However, Multi-LoRAs directly leverage fine-tuned domain adapters, assuming that domain information is available during inference. (2) To fully leverage the benefits of fine-tuned domain-specific LoRAs, we explored various approaches for the adapter weight generator. Utilizing k-means for domain adapter representation learning outperforms random sampling followed by averaging embeddings for LoRA representation. Furthermore, incorporating a fine-tuned text encoder with contrastive learning further enhances domain adaptation performance. (3) The dynamic fusion module plays a critical role in enhancing text generation quality. Without properly mixing LoRAs, embeddings from different adapters are simply averaged, which can negatively impact performance. By integrating all components, Sci-LoRA achieves the best results, demonstrating its effectiveness in cross-domain lay paraphrasing.

We also analyze the impact of fine-tuning a text encoder using contrastive learning by visualizing t-SNE embeddings from five datasets. Before fine-tuning, embeddings from different domains are intermingled, indicating that the pre-trained encoder does not effectively capture domain-specific features. After contrastive learning, domain separation improves, forming distinct clusters, though some overlap remains, particularly between CELLS and PLOS due to shared biomedical content. Additionally, some embeddings from different domains remain close, reflecting inherent semantic similarities, such as overlapping topics between SciTech-News and Life. More analysis is in Sec. C.

<table>
  <tr>
    <th></th>
    <th>COM</th>
    <th>LAY</th>
    <th>MP</th>
    <th>CON</th>
    <th>FLU</th>
    <th>ICC</th>
  </tr>
  <tr>
    <td>OPT</td>
    <td>3.65</td>
    <td>2.13</td>
    <td>2.69</td>
    <td>2.33</td>
    <td>2.46</td>
    <td>0.76</td>
  </tr>
  <tr>
    <td>LLaMA3</td>
    <td>3.47</td>
    <td><b>3.10</b></td>
    <td>2.73</td>
    <td>2.72</td>
    <td>2.87</td>
    <td>0.72</td>
  </tr>
  <tr>
    <td>Phi-4</td>
    <td>3.53</td>
    <td>2.77</td>
    <td>2.27</td>
    <td>2.20</td>
    <td>2.71</td>
    <td>0.75</td>
  </tr>
  <tr>
    <td>Mistral</td>
    <td>3.48</td>
    <td>2.64</td>
    <td>2.73</td>
    <td>2.93</td>
    <td>2.80</td>
    <td>0.28</td>
  </tr>
  <tr>
    <td>Mixtral</td>
    <td>3.42</td>
    <td>2.67</td>
    <td>2.72</td>
    <td>2.89</td>
    <td>2.64</td>
    <td>0.64</td>
  </tr>
  <tr>
    <td>Qwen2.5</td>
    <td>3.78</td>
    <td>2.86</td>
    <td>2.80</td>
    <td>2.77</td>
    <td>3.31</td>
    <td>0.75</td>
  </tr>
  <tr>
    <td>GPT-4o</td>
    <td>3.25</td>
    <td>2.68</td>
    <td>3.08</td>
    <td><b>3.53</b></td>
    <td>3.15</td>
    <td>0.75</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>3.82</b></td>
    <td>2.88</td>
    <td><b>3.45</b></td>
    <td>3.47</td>
    <td><b>3.40</b></td>
    <td><b>0.80</b></td>
  </tr>
</table>

Table 6: Mean human evaluation ratings 1-5 (the higher the better) of different models on all five datasets. Reported from left to right are: comprehensiveness, layness, meaning preservation, conciseness, and fluency.
Figure 2: t-SNE visualization (before contrastive learning and after contrastive learning) of text embeddings from CELLS, eLife, SciTechNews, PLOS and ENG in VTechAGP datasets.

4.2.3 Human Evaluation

Following a similar setting as (Cheng et al., 2025; Liu et al., 2024b; Li et al., 2024; Song et al., 2024), our evaluation uses a random sample of 15 abstracts from the total test split of all five datasets considering the workload. Judges are presented with both the academic abstract and generated general-audience abstracts from 8 models for each data sample in total of 120 abstracts. Using a 1-5 Likert scale, the judges are asked to rate the model output based on five criteria: comprehensiveness, layness, meaning preservation, conciseness, and fluency. The human evaluation setup is discussed in detail in Figure 3 in the Appendix.

The human evaluation results are presented in Table 6. Overall, Sci-LoRA demonstrates the best performance in comprehensiveness, meaning preservation, and fluency. Pre-trained models (i.e. Mixtral, GPT-4o) show high conciseness with minimal redundancy. However, they also miss detailed domain-specific contents due to their more generalized training paradigms. Fine-tuned models (i.e. OPT, Phi-4, etc.) can provide more comprehensive domain information but tend to produce verbose or repetitive outputs at the same time. By dynamically selecting the most relevant domain knowledge, Sci-LoRA effectively integrates domain-specific information while maintaining the highest level of meaning preservation. It also avoids generating verbose explanations and unnecessary elaborations. Fluency is primarily influenced by pre-training, as it is a general text generation metric. Since Sci-LoRA is built on Qwen2.5, it inherits high fluency from Qwen2.5. For layness, all models have some room for improvement. This challenge is particularly evident for highly specialized texts (i.e. biomedical domain text), where fully paraphrasing technical content into non-technical language without compromising meaning preservation remains difficult. We also report the intraclass correlation coefficient (ICC) – average fixed raters ICC, which is used to determine if items or subjects can be rated reliably by different raters. We observe that GPT4o, Qwen2.5, Phi-4, OPT and ours Sci-LoRA show good reliability. Mixtral and LLaMA3 show moderate reliability. Mistral shows poor reliability.

5 Conclusion

In this paper, we propose Sci-LoRA 2, a mixture of scientific LoRAs designed for cross-domain lay paraphrasing. Without requiring domain-specific information during inference, Sci-LoRA dynamically generates weights for domain-specific adapters. To enhance domain representation learning, a customized text encoder is fine-tuned using contrastive learning. Sci-LoRA then integrates these domain-specific adapters through a dynamic LoRA fusion module to facilitate cross-domain text generation. We evaluate Sci-LoRA against multiple state-of-the-art baselines across 12 different domains and 5 distinct datasets, using 10 automatic evaluation metrics as well as human assessment. Extensive experimental results demonstrate that Sci-LoRA consistently outperforms existing SOTA models in the cross-domain lay paraphrasing task.

2Code: https://github.com/gjiaying/Sci-LoRA
6 Limitations

Sci-LoRA has the following limitations currently: (1) Scaling Sci-LoRA to accommodate hundreds of different domains is difficult. The current implementation relies on PEFT 3 to load and merge LoRAs, with domain-specific LoRA weights computed dynamically during inference for each batch of input text. As the number of domains increases, inference latency is expected to grow significantly. (2) Sci-LoRA does not support unseen domains without training data, as each domain-specific adapter requires fine-tuning on its respective domain data. To address this limitation, we will explore few-shot learning techniques for low-resource domains for the future work. (3) Sci-LoRA is designed to integrate multiple LoRAs into a single base model. Our experiments have focused on Qwen2.5-7B-Instruct, as this pre-trained model has demonstrated strong inference performance. However, results may vary for different base models. We will investigate LoRA fusion across multiple pre-trained models to assess broader applicability in future work.

References

Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shaymal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Suha S Al-Thanyyan and Agil M Azmi. 2021. Automated text simplification: a survey. ACM Computing Surveys (CSUR), 54(2):1–36.

Kush Attal, Brian Ondov, and Dina Demner-Fushman. 2023. A dataset for plain language adaptation of biomedical abstracts. Scientific Data, 10(1):8.

Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.

Eric L. Buehler and Markus J. Buehler. 2024. X-lora: Mixture of low-rank adapter experts, a flexible framework for large language models with applications in protein mechanics and molecular design. APL Machine Learning, 2(2):026119.

Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. 2024. A survey on mixture of experts. arXiv preprint arXiv:2407.06204.

Ronald Cardenas, Bingsheng Yao, Dakuo Wang, and Yufang Hou. 2023. ‘don’t get too technical with me’: A discourse structure-based framework for automatic science journalism. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1186–1202, Singapore. Association for Computational Linguistics.

Ming Cheng, Jiaying Gong, Chenhan Yuan, William A Ingram, Edward Fox, and Hoda Eldardiry. 2025. VTechAGP: An academic-to-general-audience text paraphrase dataset and benchmark models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 6110–6130, Albuquerque, New Mexico. Association for Computational Linguistics.

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783.

Wenfeng Feng, Chuzhan Hao, Yuewei Zhang, Yu Han, and Hao Wang. 2024. Mixture-of-LoRAs: An efficient multitask tuning method for large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 11371–11380, Torino, Italia. ELRA and ICCL.

Rudolf Flesch. 1979. How to write plain English. University of Canterbury. Available at http://www.mang.canterbury.ac.nz/writing/guide/writing/flesch.shtml.[Retrieved 5 February 2016].

Marcio Fonseca and Shay Cohen. 2024. Can large language model summarizers adapt to diverse scientific
communication goals? In Findings of the Association for Computational Linguistics: ACL 2024, pages 8599–8618, Bangkok, Thailand. Association for Computational Linguistics.

Polydoros Giannouris, Theodoros Myridis, Tatiana Pasalı, and Grigorios Tsoumakas. 2024. Plain language summarization of clinical trials. In Proceedings of the Workshop on DeTermIt! Evaluating Text Difficulty in a Multilingual Context @ LREC-COLING 2024, pages 60–67, Torino, Italia. ELRA and ICCL.

Tomas Goldsack, Carolina Scarton, Matthew Shardlow, and Chenghua Lin. 2024. Overview of the BioLay-Summ 2024 shared task on the lay summarization of biomedical research articles. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 122–131, Bangkok, Thailand. Association for Computational Linguistics.

Tomas Goldsack, Zhihao Zhang, Chenghua Lin, and Carolina Scarton. 2022. Making science simple: Corpora for the lay summarisation of scientific literature. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10589–10604, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.

Jiang Guo, Darsh Shah, and Regina Barzilay. 2018. Multi-source domain adaptation with mixture of experts. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4694–4703, Brussels, Belgium. Association for Computational Linguistics.

Yue Guo, Wei Qiu, Gondy Leroy, Sheng Wang, and Trevor Cohen. 2024. Retrieval augmentation of large language models for lay language generation. Journal of Biomedical Informatics, 149:104580.

Yue Guo, Wei Qiu, Yizhong Wang, and Trevor Cohen. 2021. Automated lay language summarization of biomedical scientific reviews. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 160–168.

Zeyu Han, Chao Gao, Jinyang Liu, Jeff Zhang, and Sai Qian Zhang. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608.

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR.

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.

Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. 2023. Lorahub: Efficient cross-task generalization via dynamic lora composition. arXiv preprint arXiv:2307.13269.

Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825.

Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088.

Yuchen Jiang, Tianyu Liu, Shuming Ma, Dongdong Zhang, Jian Yang, Haoyang Huang, Rico Sennrich, Ryan Cotterell, Mrinmaya Sachan, and Ming Zhou. 2022. BlonDe: An automatic evaluation metric for document-level machine translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1550–1565, Seattle, United States. Association for Computational Linguistics.

Hwanmun Kim, Kamal Raj Kanakarajan, and Malaikanan Sankarasubbu. 2024. Saama technologies at biolaysumm: Abstract based fine-tuned models with lora. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 786–792.

Hao Li, Yuping Wu, Viktor Schlegel, Riza Batista-Navarro, Tharindu Madusanka, Iqra Zahid, Jiayan Zeng, Xiaochi Wang, Xinran He, Yizhi Li, and Goran Nenadic. 2024. Which side are you on? a multi-task dataset for end-to-end argument summarisation and evaluation. In Findings of the Association for Computational Linguistics ACL 2024, pages 133–150, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.

Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.

Chin-Yew Lin and Franz Josef Och. 2004. ORANGE: A method for evaluating automatic evaluation metrics for machine translation. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pages 501–507, Geneva, Switzerland. COLING.

Dongqi Liu, Yifan Wang, and Vera Demberg. 2023. Incorporating distributions of discourse structure for long document abstractive summarization. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5574–5590, Toronto, Canada. Association for Computational Linguistics.

Jialin Liu, Jianhua Wu, Jie Liu, and Yutai Duan. 2024a. Learning attentional mixture of loras for
language model continual learning. arXiv preprint arXiv:2409.19611.

Ran Liu, Ming Liu, Min Yu, He Zhang, Jianguo Jiang, Gang Li, and Weiqing Huang. 2024b. SumSurvey: An abstractive dataset of scientific survey papers for long document summarization. In Findings of the Association for Computational Linguistics ACL 2024, pages 9632–9651, Bangkok, Thailand and virtual meeting. Association for Computational Linguistics.

Yuhuan Liu, Shangbin Feng, Xiaochuang Han, Vidhisha Balachandran, Chan Young Park, Sachin Kumar, and Yulia Tsvetkov. 2024c. P^3Sum: Preserving author’s perspective in news summarization with diffusion language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2154–2173, Mexico City, Mexico. Association for Computational Linguistics.

Tongxu Luo, Jiahe Lei, Fangyu Lei, Weihao Liu, Shizhu He, Jun Zhao, and Kang Liu. 2024. Moelora: Contrastive learning guided mixture of experts on parameter-efficient fine-tuning for large language models. arXiv preprint arXiv:2402.12851.

Chuancheng Lv, Lei Li, Shitou Zhang, Gang Chen, Fan-chao Qi, Ningyu Zhang, and Hai-Tao Zheng. 2024. HyperLoRA: Efficient cross-task generalization via constrained low-rank adapters generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 16376–16393, Miami, Florida, USA. Association for Computational Linguistics.

Hemang Malik, Gaurav Pradeep, and Pratinav Seth. 2024. HGP-NLP at BioLaySumm: Leveraging LoRA for lay summarization of biomedical research articles using Seq2Seq transformers. In Proceedings of the 23rd Workshop on Biomedical Natural Language Processing, pages 831–836, Bangkok, Thailand. Association for Computational Linguistics.

Mohammed Muqeeth, Haokun Liu, Yufan Liu, and Colin Raffel. 2024. Learning to route among specialized experts for zero-shot generalization. arXiv preprint arXiv:2402.05859.

Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748.

Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi. 2024. Lora soups: Merging loras for practical skill composition tasks. arXiv preprint arXiv:2410.13025.

Ricardo Rei, Ana C Farinha, José G.C. de Souza, Pedro G. Ramos, André F.T. Martins, Luisa Coheur, and Alon Lavie. 2022. Searching for COMETINHO: The little metric that could. In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation, pages 61–70, Ghent, Belgium. European Association for Machine Translation.

Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics.

Hwanjun Song, Hang Su, Igor Shalyminov, Jason Cai, and Saab Mansour. 2024. FineSurE: Fine-grained summarization evaluation using LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 906–922, Bangkok, Thailand. Association for Computational Linguistics.

Chen Tang, Shun Wang, Tomas Goldsack, and Chenghua Lin. 2023. Improving biomedical abstractive summarisation with knowledge aggregation from citation papers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 606–618, Singapore. Association for Computational Linguistics.

Qwen Team. 2024. Qwen2.5: A party of foundation models.

Xun Wu, Shaohan Huang, and Furu Wei. 2024. Mixture of IoRA experts. In The Twelfth International Conference on Learning Representations.

Jingwei Xu, Junyu Lai, and Yunpeng Huang. 2024. Meteora: Multiple-tasks embedded lora for large language models. arXiv preprint arXiv:2405.13053.

Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. Optimizing statistical machine translation for text simplification. Transactions of the Association for Computational Linguistics, 4:401–415.

Prateek Yadav, Derek Tam, Leshem Choshen, Colin A Raffel, and Mohit Bansal. 2024. Ties-merging: Resolving interference when merging models. Advances in Neural Information Processing Systems, 36.

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2024. Language models are super mario: Absorbing abilities from homologous models as a free lunch. In Forty-first International Conference on Machine Learning.

Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher DeWan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068.

Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. BERTscore: Evaluating text generation with BERT. In International Conference on Learning Representations.
Yuxuan Zhang and Ruizhe Li. 2024. Dlp-lora: Efficient task-specific lora fusion with a dynamic, lightweight plugin for large language models. arXiv preprint arXiv:2410.01497.

Lulu Zhao, Weihao Zeng, Xiaofeng Shi, and Hua Zhou. 2024a. Mosld: An extremely parameter-efficient mixture-of-shared loras for multi-task learning. arXiv preprint arXiv:2412.08946.

Ziyu Zhao, Leilei Gan, Guoyin Wang, Wangchunshu Zhou, Hongxia Yang, Kun Kuang, and Fei Wu. 2024b. LoraRetriever: Input-aware LoRA retrieval and composition for mixed tasks in the wild. In Findings of the Association for Computational Linguistics: ACL 2024, pages 4447–4462, Bangkok, Thailand. Association for Computational Linguistics.

Ziyu Zhao, Tao Shen, Didi Zhu, Zexi Li, Jing Su, Xuwu Wang, Kun Kuang, and Fei Wu. 2024c. Merging loras like playing lego: Pushing the modularity of lora to extremes through rank-wise clustering. arXiv preprint arXiv:2409.16167.

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyuan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand. Association for Computational Linguistics.

Jianing Zhou and Suma Bhat. 2021. Paraphrase generation: A survey of the state of the art. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 5075–5086.

A More Related Works

Text paraphrasing aims at rewriting text by different words or sentence structures while keeping the original meaning (Zhou and Bhat, 2021). Text simplification modifies syntax and lexicon to improve the understandability of language for young readers (Al-Thanyyan and Azmi, 2021). Lay summarization involves generating a simplified summary of a technical or specialist text that is suitable for a non-expert audience (Goldsack et al., 2024; Giannouris et al., 2024). Lay paraphrasing focuses on rewriting the text written from the technical experts to the general audience without specialized domain knowledge (Cheng et al., 2025; Guo et al., 2024). Table 7 shows examples of differences for the above four tasks.

B Dataset Analysis

In our experiments, we evaluate our model over five widely used public datasets: PLOS (Goldsack et al., 2022), eLife (Goldsack et al., 2022), CELLS (Guo et al., 2024), SciTechNews (Cardenas et al., 2023), and VTechAGP (Cheng et al., 2025). We followed the original training, validation and testing splitting for PLOS, eLife, CELLS and VTechAGP. For SciTechNew, because the training set does not have abstract pairs while the validation and testing sets have the scientific and non-technical paraphrased pairs, we randomly resplit the dataset from original validation and testing sets into training, validation and testing sets by 0.8:0.1:0.1. In the following, we describe each dataset in detail.

• PLOS (Goldsack et al., 2022) consists of abstracts from biomedical articles paired with non-technical lay summaries in science and medicine domain sourced from the peer-reviewed journals of The Public Library of Science publisher.

• eLife (Goldsack et al., 2022) dataset contains scientific abstracts paired with non-technical lay summaries in the field of biomedical and life sciences derived from an open-access peer-reviewed journal.

• CELLS (Guo et al., 2024) is the paragraph-paired dataset of scientific abstracts and expert-authored plain language summaries in the biomedicine field derived from biomedical journals for the lay language generation task.

• SciTechNews (Cardenas et al., 2023) is a text-to-text science journalism dataset consisting of scientific papers paired with their corresponding press release snippet mined from ACM TechNews about scientific achievements and technology.

• VTechAGP (Cheng et al., 2025) is an academic-to-general-audience text paraphrase dataset derived from electronic theses and dissertations across eight different colleges sourced from Virginia Tech Graduate School and Digital Libraries.

Following (Cheng et al., 2025), we report some basic dataset statistics in Table 2, including the number of documents for each public dataset, the average number of sentences for each document, and the average sentence length for the source and target texts, respectively. We also report Flesch-Kincaid Grade Level (FKGL) and Dale-Chall Readability Score (DCRS) follows (Goldsack et al., 2022). Both FKGL and DCRS estimate the U.S.
grade level required to comprehend a given text. FKGL calculates this based on the total count of sentences, words, and syllables in the text. In contrast, DCRS evaluates readability by considering the average sentence length and the number of familiar words, referencing a lookup table of the 3,000 most commonly used English words.

C Ablation Study

Table 8 and Table 9 present the complete results of the ablation study on Sci-LoRA’s components across 12 scientific domains over five distinct datasets. From both tables, we observe that fine-tuning pre-trained models with LoRA (both Multi-LoRAs and Single LoRA) significantly enhances performance. In most cases, fine-tuning a dedicated LoRA for a specific domain yields better results than using a single LoRA across all domains. However, there are notable exceptions. For instance, on the eLife and SciTechNews datasets, a single LoRA outperforms multiple domain-specific LoRAs. This is likely due to the relatively small training dataset sizes for these domains, making it challenging to learn robust representations with limited data. Additionally, some topics in the eLife dataset are closely related to the biomedical domain, which aligns with CELLS—the largest training dataset. As a result, fine-tuning a single LoRA across all datasets allows it to leverage knowledge from CELLS, leading to improved performance.

All components of Sci-LoRA—the k-means-based adapter weight generator, the contrastive-learning-trained text encoder, and the dynamic fusion module—play crucial roles in its effectiveness. When integrated, these components enable Sci-LoRA to consistently outperform other methods across multiple key metrics. Notably, it achieves the highest scores in sentence BLEU, document BLEU, BERTScore, ROUGE-1, ROUGE-2, METEOR, and SARI across most domains, demonstrating that domain-specific fine-tuning significantly enhances text generation quality. One notable observation is that for COMET, an end-to-end evaluation metric, pre-trained models consistently yield the best performance. Further exploration into improving Sci-LoRA’s end-to-end performance remains an avenue for future research.

To analyze the impact of the fine-tuned text encoder using contrastive learning as discussed in Sec. 3.2.2, we present t-SNE visualizations of the text embeddings generated by the encoder both before and after contrastive learning. The embeddings are from five datasets: CELLS, eLife, SciTechNews, PLOS, and ENG in VTechAGP. For better demonstration, we select the largest domain ENG from VTechAGP for visualization. As shown in Figure 2, the data points corresponding to different domains (CELLs, eLife, SciTechNews, PLOS, ENG) are intermingled. There is no clear separation between the domains, indicating that the embeddings produced by the pre-trained text encoder (without contrastive learning) do not capture domain-specific features effectively. The central point (average embedding) for each domain is shown as a star in Figure 2. The closeness of points from different domains indicates that embeddings for different domains are overly similar, making it difficult to differentiate between them.

After using contrastive learning for fine-tuning the text encoder as discussed in Sec. 3.2.2, the embeddings exhibit much clearer separations between domains. Each domain forms a distinct cluster, demonstrating that the contrastive learning process successfully pushed embeddings from different domains apart while pulling embeddings within the same domain closer. Note that embeddings of PLOS dataset (red points) and embeddings of CELLS dataset (blue points) are still intermingled after contrastive learning. This is because as discussed in CELLS (Guo et al., 2024), CELLS dataset has an overlap with PLOS dataset (Gold-sack et al., 2022), and text in both datasets are mainly from biomedical domain. In addition, there still exist text embeddings from different domains that are intermingled (overlapped data points with different colors). This is because the data may inherently have overlapping features across domains. For example, some text samples may contain content relevant to both the green cluster (eLife domain) and the orange cluster (SciTechNews domain), causing their embeddings to appear closer in the high-dimensional space. This also validates that there may be semantic similarities between certain domains. For instance, both SciTechNews and eLife could have overlapping topics such as scientific advancements, which makes their embeddings similar and distance closer.
Human Evaluation Guideline

This guideline is intended to give annotators a clear understanding of the task and requirements before manual evaluation. Be sure to read the following content carefully.

This task is used to assess the quality of general-audience abstract generated by different models. You are required to complete 15 subtasks in total, each of which will provide you with the original academic abstract and 8 general-audience abstracts generated by different models. You need to score each generated abstracts based on five evaluation indicators, with score of 1 represents the worst and 5 represents the bests. The five evaluation indicators are:

1. Comprehensiveness/Completeness: to what extent does the model output contain all information that might be necessary for a non-expert to understand the high-level topic of the article and significance of the research.
2. Layness: to what extent is the content of the model output comprehensible (or readable) to a non-expert, in terms of both structure and language.
3. Factuality/Correctness/Meaning Preservation: to what extent is the model output factually consistent and grammatically correct.
4. Conciseness/Non-Redundancy: to what extent does the model output contain non-redundant or non-repeated information.
5. Fluency/Coherence: to what extent does the model output read smoothly and naturally, without grammatical, spelling or formatting errors. Sentences should be coherent and consistent with natural reading habits, rather than simply stacking sentences together. Both quality of individual sentence and relationships between sentences should be considered.

Please note that you will not know the eight abstracts is generated by which model respectively, and their order in different subtasks is random.

Evaluation results are only used for this study. All the information will be anonymized, and your personal preferences will not be disclosed. You do not have to bear any responsibility for the risk caused by your evaluation results.

Figure 3: Human evaluation guideline.
Table 7: An example of the differences among text paraphrasing, text simplification, lay summarization and lay paraphrasing. The example input text is from VTechAGP dataset and results are generated by ChatGPT.

<table>
  <tr>
    <th>Input Text</th>
    <td>Thin filament pyrometry is a proven technique used to measure flame temperature by capturing the spectral radiance produced by the immersion of silicon carbide filaments in a hot gas environment. In this study a commercially available CMOS color camera was used, and the spectral response of each color channel was integrated with respect to the assumed graybody radiation spectrum to form a look up table between color ratio and temperature. Interpolated filament temperatures are then corrected for radiation losses via an energy balance to determine the flame temperature. Verification of the technique was performed on the Holthuis and Associates Flat Flame Burner, formerly known as the Mckenna Burner, and the results are directly compared to literature values measured on a similar burner. The results are also supported by radiation corrected measurements taken using a type B thermocouple on the same burner setup. An error propagation analysis was performed to determine which factors contribute the most to the final measurement uncertainty and confidence intervals are calculated for the results. Uncertainty values for a single point measurement were determined to be between ±15 and ±50 K depending on the color ratio and the total uncertainty associated with day-to-day changes in the measurement setup was found to be ±55 K.</td>
  </tr>
  <tr>
    <th>Text Paraphrasing</th>
    <td>Thin filament pyrometry is an established method for measuring flame temperatures by analyzing the spectral radiance emitted when silicon carbide filaments are immersed in a hot gas. In this study, a commercially available CMOS color camera was utilized, and the spectral response of each color channel was integrated with the assumed graybody radiation spectrum to create a lookup table correlating color ratio to temperature. The filament temperatures obtained through interpolation were adjusted for radiation losses using an energy balance to determine the flame temperature. The method was validated using the Holthuis and Associates Flat Flame Burner (previously known as the Mckenna Burner), and the results were directly compared with literature values from similar burners. Additionally, radiation-corrected measurements from a type B thermocouple on the same setup supported the findings. An error propagation analysis identified the key contributors to measurement uncertainty, and confidence intervals were calculated. The uncertainty for a single measurement ranged from ±15 to ±50 K, depending on the color ratio, while day-to-day variations in the setup introduced an uncertainty of ±55 K.</td>
  </tr>
  <tr>
    <th>Text Simplification</th>
    <td>Thin filament pyrometry is a reliable method for measuring flame temperature by analyzing the light emitted when silicon carbide filaments are placed in hot gas. In this study, a CMOS color camera was used, and a table was created to link color ratios to temperature based on the assumed radiation spectrum. Filament temperatures were adjusted for radiation losses to calculate the flame temperature. The method was tested on a Flat Flame Burner, also known as the Mckenna Burner, and the results matched previously published data. Additional tests using a type B thermocouple confirmed the findings. An analysis of errors showed which factors affected measurement accuracy the most, with uncertainties ranging from ±15 to ±50 K for single measurements and ±55 K for variations in the setup over time.</td>
  </tr>
  <tr>
    <th>Lay Summarization</th>
    <td>Thin filament pyrometry is a technique used to measure how hot a flame is by analyzing the light given off by a small filament placed in the flame. In this study, researchers used a camera to measure light colors and linked these to temperatures using a pre-made chart. They adjusted the results to account for energy lost as heat and tested the method on a standard burner. The findings matched results from other studies and were confirmed with another temperature-measuring tool. The researchers also identified factors that affect accuracy and found that temperature measurements could vary by about 15 to 55 K due to setup changes.</td>
  </tr>
  <tr>
    <th>Lay Paraphrasing</th>
    <td>Thin filament pyrometry is a method used to measure flame temperatures by analyzing the light emitted from a small silicon carbide filament placed in hot gas. In this study, researchers used a regular CMOS color camera to capture light colors and created a chart to link color ratios to temperatures based on expected radiation patterns. They adjusted the temperature measurements to account for heat loss and tested their method on a standard burner setup, known as the Flat Flame Burner or Mckenna Burner. The results matched previous studies and were confirmed with another tool called a type B thermocouple. The researchers also examined what factors caused measurement errors and calculated that single temperature readings could vary by 15 to 50 K, while changes in the setup added an uncertainty of about 55 K.</td>
  </tr>
</table>
Table 8: Full ablation study over Sci-LoRA components across 12 domains in all five datasets. (Part I)

<table>
  <tr>
    <th></th>
    <th>s-BLEU</th>
    <th>d-BLEU</th>
    <th>BERTScore</th>
    <th>BLONDE</th>
    <th>ROUGE1</th>
    <th>ROUGE2</th>
    <th>METEOR</th>
    <th>COMET</th>
    <th>SARI</th>
    <th>FRES</th>
  </tr>
  <tr>
    <th colspan="11">CELLS</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>2.34</td>
    <td>5.41</td>
    <td>81.80</td>
    <td>16.83</td>
    <td>37.84</td>
    <td>9.67</td>
    <td>32.79</td>
    <td><b>81.55</b></td>
    <td>40.32</td>
    <td>34.66</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>3.11</td>
    <td>9.51</td>
    <td>82.35</td>
    <td><b>19.09</b></td>
    <td>40.55</td>
    <td>11.95</td>
    <td>29.84</td>
    <td>78.77</td>
    <td>40.43</td>
    <td>43.53</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>3.07</td>
    <td>9.26</td>
    <td>82.36</td>
    <td>18.80</td>
    <td>40.55</td>
    <td>11.92</td>
    <td>30.40</td>
    <td>78.95</td>
    <td>40.38</td>
    <td>43.43</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>2.48</td>
    <td>6.83</td>
    <td>81.33</td>
    <td>14.37</td>
    <td>36.85</td>
    <td>9.92</td>
    <td>27.78</td>
    <td>77.82</td>
    <td>40.16</td>
    <td>43.63</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>2.41</td>
    <td>6.58</td>
    <td>81.30</td>
    <td>15.26</td>
    <td>36.74</td>
    <td>9.79</td>
    <td>27.99</td>
    <td>78.11</td>
    <td>40.24</td>
    <td>43.73</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>2.56</td>
    <td>7.09</td>
    <td>81.49</td>
    <td>16.42</td>
    <td>37.38</td>
    <td>10.08</td>
    <td>27.98</td>
    <td>77.92</td>
    <td>40.33</td>
    <td>43.53</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>2.09</td>
    <td>6.24</td>
    <td>80.66</td>
    <td>15.71</td>
    <td>37.54</td>
    <td>9.88</td>
    <td>27.23</td>
    <td>77.59</td>
    <td>40.86</td>
    <td>44.14</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>3.99</b></td>
    <td><b>11.15</b></td>
    <td><b>83.00</b></td>
    <td>18.12</td>
    <td><b>43.10</b></td>
    <td><b>14.02</b></td>
    <td><b>32.29</b></td>
    <td>79.10</td>
    <td><b>41.15</b></td>
    <td><b>44.36</b></td>
  </tr>
  <tr>
    <th colspan="11">PLOS</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>2.44</td>
    <td>6.18</td>
    <td>82.44</td>
    <td>17.80</td>
    <td>40.65</td>
    <td>10.67</td>
    <td>33.12</td>
    <td><b>81.68</b></td>
    <td>40.03</td>
    <td>34.36</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>3.09</td>
    <td>10.15</td>
    <td>82.68</td>
    <td>19.58</td>
    <td>42.69</td>
    <td>13.25</td>
    <td>31.78</td>
    <td>79.88</td>
    <td>40.24</td>
    <td>43.43</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>3.23</td>
    <td>10.18</td>
    <td>82.70</td>
    <td><b>21.46</b></td>
    <td>42.75</td>
    <td>13.25</td>
    <td>31.88</td>
    <td><b>79.90</b></td>
    <td><b>40.08</b></td>
    <td>43.43</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>2.23</td>
    <td>6.24</td>
    <td>81.13</td>
    <td>14.63</td>
    <td>37.37</td>
    <td>10.00</td>
    <td>28.88</td>
    <td>78.75</td>
    <td>39.46</td>
    <td>45.25</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>2.35</td>
    <td>6.38</td>
    <td>81.20</td>
    <td>13.90</td>
    <td>37.45</td>
    <td>10.13</td>
    <td>29.23</td>
    <td>78.95</td>
    <td>39.62</td>
    <td><b>45.46</b></td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>2.39</td>
    <td>6.57</td>
    <td>81.41</td>
    <td>16.64</td>
    <td>37.97</td>
    <td>10.22</td>
    <td>28.54</td>
    <td>78.66</td>
    <td>39.62</td>
    <td>44.95</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>2.52</td>
    <td>6.25</td>
    <td>80.85</td>
    <td>14.56</td>
    <td>42.32</td>
    <td>10.97</td>
    <td>33.58</td>
    <td>79.53</td>
    <td>40.03</td>
    <td>44.24</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>4.06</b></td>
    <td><b>12.43</b></td>
    <td><b>83.35</b></td>
    <td>16.81</td>
    <td><b>45.59</b></td>
    <td><b>15.89</b></td>
    <td><b>35.20</b></td>
    <td>80.29</td>
    <td>40.07</td>
    <td>44.36</td>
  </tr>
  <tr>
    <th colspan="11">eLife</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>0.85</td>
    <td>3.17</td>
    <td>80.52</td>
    <td><b>5.00</b></td>
    <td>36.19</td>
    <td>8.27</td>
    <td>20.36</td>
    <td>79.44</td>
    <td>42.67</td>
    <td>42.51</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>1.19</td>
    <td>3.99</td>
    <td>80.48</td>
    <td>4.04</td>
    <td>37.37</td>
    <td>9.05</td>
    <td>21.31</td>
    <td>78.84</td>
    <td>43.84</td>
    <td>53.21</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td><b>1.22</b></td>
    <td>3.98</td>
    <td>80.72</td>
    <td>4.97</td>
    <td>37.57</td>
    <td>9.23</td>
    <td>21.21</td>
    <td>79.09</td>
    <td>44.03</td>
    <td>53.41</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>1.12</td>
    <td>6.25</td>
    <td>81.24</td>
    <td>4.36</td>
    <td>41.95</td>
    <td>10.10</td>
    <td>27.66</td>
    <td>81.82</td>
    <td>47.39</td>
    <td><b>55.13</b></td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>1.17</td>
    <td>6.52</td>
    <td>81.39</td>
    <td>4.39</td>
    <td>42.38</td>
    <td>10.31</td>
    <td>27.92</td>
    <td>82.06</td>
    <td>47.63</td>
    <td>54.93</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>1.19</td>
    <td><b>6.58</b></td>
    <td><b>81.62</b></td>
    <td>4.89</td>
    <td>42.22</td>
    <td>10.41</td>
    <td>28.40</td>
    <td>82.54</td>
    <td><b>47.80</b></td>
    <td>54.83</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>0.93</td>
    <td>3.11</td>
    <td>81.13</td>
    <td>4.06</td>
    <td>38.50</td>
    <td>9.28</td>
    <td>23.07</td>
    <td>81.76</td>
    <td>44.11</td>
    <td>54.35</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>1.22</b></td>
    <td>6.09</td>
    <td>81.40</td>
    <td>4.99</td>
    <td><b>42.59</b></td>
    <td><b>11.31</b></td>
    <td><b>28.98</b></td>
    <td><b>82.95</b></td>
    <td>47.64</td>
    <td>54.93</td>
  </tr>
  <tr>
    <th colspan="11">SciTechNews</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>0.95</td>
    <td>2.82</td>
    <td>77.37</td>
    <td>9.07</td>
    <td>30.30</td>
    <td>5.10</td>
    <td><b>24.43</b></td>
    <td><b>76.29</b></td>
    <td>39.15</td>
    <td>34.76</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>1.84</td>
    <td>4.09</td>
    <td>78.00</td>
    <td>8.74</td>
    <td>31.93</td>
    <td>5.92</td>
    <td>22.89</td>
    <td>74.35</td>
    <td>43.35</td>
    <td>41.29</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>1.58</td>
    <td>4.26</td>
    <td>77.90</td>
    <td>9.10</td>
    <td>31.85</td>
    <td>6.23</td>
    <td>22.90</td>
    <td>74.32</td>
    <td>43.02</td>
    <td><b>42.11</b></td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>2.03</td>
    <td>4.09</td>
    <td>78.01</td>
    <td>9.04</td>
    <td>31.60</td>
    <td>6.11</td>
    <td>21.79</td>
    <td>73.64</td>
    <td>43.66</td>
    <td>41.09</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>2.07</td>
    <td>4.11</td>
    <td>78.83</td>
    <td>9.60</td>
    <td>31.73</td>
    <td>6.79</td>
    <td>22.99</td>
    <td>73.81</td>
    <td>43.52</td>
    <td>41.19</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>2.19</td>
    <td>4.17</td>
    <td><b>78.88</b></td>
    <td>9.89</td>
    <td>32.29</td>
    <td>6.88</td>
    <td>22.21</td>
    <td>75.15</td>
    <td>43.42</td>
    <td>41.29</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>1.18</td>
    <td>4.03</td>
    <td>78.30</td>
    <td>8.91</td>
    <td>30.39</td>
    <td>6.22</td>
    <td>23.09</td>
    <td>74.36</td>
    <td>42.88</td>
    <td>38.26</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>2.20</b></td>
    <td><b>4.61</b></td>
    <td>78.82</td>
    <td><b>10.13</b></td>
    <td><b>32.68</b></td>
    <td><b>6.90</b></td>
    <td>23.87</td>
    <td>76.00</td>
    <td><b>43.76</b></td>
    <td>41.38</td>
  </tr>
  <tr>
    <th colspan="11">College of Agriculture and Life Sciences (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>3.92</td>
    <td>13.04</td>
    <td>84.15</td>
    <td>10.71</td>
    <td>48.78</td>
    <td>16.96</td>
    <td>35.58</td>
    <td><b>85.46</b></td>
    <td>36.91</td>
    <td>34.97</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>8.73</td>
    <td>27.00</td>
    <td>84.97</td>
    <td>16.05</td>
    <td>52.49</td>
    <td>27.97</td>
    <td>41.29</td>
    <td>82.88</td>
    <td>40.12</td>
    <td>34.76</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>8.95</td>
    <td>25.55</td>
    <td>84.98</td>
    <td>36.02</td>
    <td>52.90</td>
    <td>27.50</td>
    <td>40.88</td>
    <td>83.42</td>
    <td>40.14</td>
    <td>35.07</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>9.01</td>
    <td>26.61</td>
    <td>84.77</td>
    <td>36.92</td>
    <td>52.52</td>
    <td>26.97</td>
    <td>40.51</td>
    <td>82.65</td>
    <td>39.72</td>
    <td>34.36</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>9.26</td>
    <td>26.67</td>
    <td>84.92</td>
    <td>35.90</td>
    <td>53.10</td>
    <td>27.30</td>
    <td>40.96</td>
    <td>82.84</td>
    <td>40.45</td>
    <td>42.82</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>9.91</td>
    <td>26.83</td>
    <td>84.83</td>
    <td>36.41</td>
    <td>53.20</td>
    <td>27.93</td>
    <td>40.91</td>
    <td>82.97</td>
    <td>40.72</td>
    <td>42.56</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>6.18</td>
    <td>24.67</td>
    <td>84.47</td>
    <td>22.22</td>
    <td>52.74</td>
    <td>27.39</td>
    <td>40.56</td>
    <td>82.66</td>
    <td>40.38</td>
    <td>40.86</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>10.21</b></td>
    <td><b>31.03</b></td>
    <td><b>86.01</b></td>
    <td><b>36.99</b></td>
    <td><b>56.90</b></td>
    <td><b>32.16</b></td>
    <td><b>45.51</b></td>
    <td>83.79</td>
    <td><b>41.32</b></td>
    <td><b>44.86</b></td>
  </tr>
  <tr>
    <th colspan="11">College of Architecture, Arts, and Design (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>5.63</td>
    <td>12.69</td>
    <td>81.97</td>
    <td>27.56</td>
    <td>42.46</td>
    <td>13.87</td>
    <td>33.98</td>
    <td><b>80.05</b></td>
    <td>34.64</td>
    <td>35.17</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>19.78</td>
    <td>34.39</td>
    <td>83.30</td>
    <td>46.43</td>
    <td>47.83</td>
    <td>26.51</td>
    <td>41.77</td>
    <td>77.20</td>
    <td>40.76</td>
    <td>42.41</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>13.43</td>
    <td>28.71</td>
    <td>82.55</td>
    <td>40.64</td>
    <td>46.51</td>
    <td>23.37</td>
    <td>39.85</td>
    <td>76.06</td>
    <td>39.39</td>
    <td><b>51.89</b></td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>23.08</td>
    <td>34.74</td>
    <td>83.46</td>
    <td>48.14</td>
    <td>48.81</td>
    <td>26.16</td>
    <td>40.48</td>
    <td>77.30</td>
    <td>41.58</td>
    <td>42.41</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td><b>23.17</b></td>
    <td>37.08</td>
    <td>83.72</td>
    <td>50.02</td>
    <td>51.13</td>
    <td>28.28</td>
    <td>43.91</td>
    <td>77.83</td>
    <td>43.56</td>
    <td>42.31</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>18.06</td>
    <td>38.62</td>
    <td>84.03</td>
    <td>50.00</td>
    <td>52.07</td>
    <td>30.86</td>
    <td>44.89</td>
    <td>77.97</td>
    <td>44.00</td>
    <td>41.29</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>14.86</td>
    <td>31.22</td>
    <td>81.59</td>
    <td>44.02</td>
    <td>50.82</td>
    <td>22.62</td>
    <td>41.42</td>
    <td>78.59</td>
    <td>43.63</td>
    <td>44.03</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>18.38</td>
    <td><b>38.97</b></td>
    <td><b>84.37</b></td>
    <td><b>50.99</b></td>
    <td><b>52.69</b></td>
    <td><b>30.98</b></td>
    <td><b>46.71</b></td>
    <td>78.02</td>
    <td><b>44.26</b></td>
    <td>41.80</td>
  </tr>
</table>
Table 9: Full ablation study over Sci-LoRA components across 12 domains in all five datasets. (Part II)

<table>
  <tr>
    <th></th>
    <th>s-BLEU</th>
    <th>d-BLEU</th>
    <th>BERTScore</th>
    <th>BLONDE</th>
    <th>ROUGE1</th>
    <th>ROUGE2</th>
    <th>METEOR</th>
    <th>COMET</th>
    <th>SARI</th>
    <th>FRES</th>
  </tr>
  <tr>
    <th colspan="11">College of Engineering (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>3.57</td>
    <td>10.74</td>
    <td>82.88</td>
    <td>6.39</td>
    <td>44.78</td>
    <td>14.45</td>
    <td>34.85</td>
    <td><b>83.84</b></td>
    <td>35.85</td>
    <td>26.20</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>9.72</td>
    <td>24.45</td>
    <td>83.57</td>
    <td>9.52</td>
    <td>48.39</td>
    <td>23.38</td>
    <td>39.08</td>
    <td>82.02</td>
    <td>38.73</td>
    <td>32.63</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>9.72</td>
    <td>24.40</td>
    <td>83.16</td>
    <td>9.27</td>
    <td>47.54</td>
    <td>23.20</td>
    <td>38.47</td>
    <td>81.22</td>
    <td>38.74</td>
    <td>32.43</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>11.13</td>
    <td>25.18</td>
    <td>83.72</td>
    <td>9.44</td>
    <td>48.74</td>
    <td>24.60</td>
    <td>39.79</td>
    <td>81.69</td>
    <td>40.55</td>
    <td>32.43</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>11.60</td>
    <td>25.77</td>
    <td>83.59</td>
    <td>9.74</td>
    <td>48.74</td>
    <td>24.87</td>
    <td>39.99</td>
    <td>81.56</td>
    <td>41.32</td>
    <td>32.33</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td><b>11.69</b></td>
    <td>25.95</td>
    <td>83.92</td>
    <td>9.94</td>
    <td>49.54</td>
    <td>24.94</td>
    <td>40.59</td>
    <td>82.25</td>
    <td><b>41.64</b></td>
    <td>33.33</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>9.33</td>
    <td>23.74</td>
    <td>83.86</td>
    <td>9.69</td>
    <td>46.52</td>
    <td>22.16</td>
    <td>38.78</td>
    <td>82.81</td>
    <td>38.48</td>
    <td><b>33.85</b></td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>11.69</b></td>
    <td><b>28.31</b></td>
    <td><b>84.30</b></td>
    <td><b>10.28</b></td>
    <td><b>51.45</b></td>
    <td><b>27.62</b></td>
    <td><b>42.73</b></td>
    <td>82.90</td>
    <td><b>41.64</b></td>
    <td>33.82</td>
  </tr>
  <tr>
    <th colspan="11">College of Liberal Arts and Human Sciences (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>6.95</td>
    <td>14.62</td>
    <td>84.35</td>
    <td>30.19</td>
    <td>48.16</td>
    <td>19.00</td>
    <td>39.44</td>
    <td><b>83.42</b></td>
    <td>33.34</td>
    <td>33.34</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>20.38</td>
    <td>37.63</td>
    <td>86.30</td>
    <td>45.29</td>
    <td>56.44</td>
    <td>36.19</td>
    <td>49.57</td>
    <td>81.86</td>
    <td>40.75</td>
    <td>30.80</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>19.82</td>
    <td>36.59</td>
    <td>86.24</td>
    <td>46.14</td>
    <td>56.03</td>
    <td>35.59</td>
    <td>49.26</td>
    <td>81.86</td>
    <td>38.98</td>
    <td>30.50</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>23.26</td>
    <td>35.87</td>
    <td>86.35</td>
    <td>45.44</td>
    <td>56.20</td>
    <td>36.04</td>
    <td>49.16</td>
    <td>81.89</td>
    <td>40.20</td>
    <td>30.20</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>23.44</td>
    <td>38.47</td>
    <td>86.66</td>
    <td><b>48.92</b></td>
    <td>57.45</td>
    <td>37.46</td>
    <td>51.10</td>
    <td>82.20</td>
    <td>42.26</td>
    <td>30.09</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>23.92</td>
    <td>38.71</td>
    <td>86.62</td>
    <td>48.83</td>
    <td>57.93</td>
    <td>37.95</td>
    <td>51.41</td>
    <td>82.56</td>
    <td>42.56</td>
    <td>30.90</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>18.07</td>
    <td>37.69</td>
    <td>86.31</td>
    <td>42.23</td>
    <td>56.96</td>
    <td>35.41</td>
    <td>49.37</td>
    <td>82.44</td>
    <td>39.72</td>
    <td>32.73</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>23.99</b></td>
    <td><b>40.33</b></td>
    <td><b>87.25</b></td>
    <td>48.32</td>
    <td><b>59.59</b></td>
    <td><b>40.92</b></td>
    <td><b>53.37</b></td>
    <td>82.62</td>
    <td><b>42.88</b></td>
    <td>32.48</td>
  </tr>
  <tr>
    <th colspan="11">College of Natural Resources and Environment (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>3.96</td>
    <td>12.24</td>
    <td>84.34</td>
    <td>23.64</td>
    <td>49.13</td>
    <td>16.96</td>
    <td>37.53</td>
    <td><b>85.81</b></td>
    <td>35.85</td>
    <td>35.27</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>11.12</td>
    <td>26.78</td>
    <td>85.11</td>
    <td>35.63</td>
    <td>53.79</td>
    <td>29.24</td>
    <td>43.89</td>
    <td>83.25</td>
    <td>40.91</td>
    <td>42.82</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>7.55</td>
    <td>24.42</td>
    <td>84.75</td>
    <td>34.18</td>
    <td>51.67</td>
    <td>26.53</td>
    <td>41.31</td>
    <td>83.23</td>
    <td>38.49</td>
    <td>42.82</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>11.66</td>
    <td>27.60</td>
    <td>85.48</td>
    <td>38.64</td>
    <td>54.67</td>
    <td>29.93</td>
    <td>45.09</td>
    <td>84.17</td>
    <td>40.82</td>
    <td>41.50</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>11.76</td>
    <td>28.07</td>
    <td>85.51</td>
    <td>38.43</td>
    <td>54.96</td>
    <td>30.21</td>
    <td>45.14</td>
    <td>84.08</td>
    <td>40.72</td>
    <td>41.80</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>11.93</td>
    <td>28.80</td>
    <td>85.71</td>
    <td>38.64</td>
    <td>54.93</td>
    <td>30.30</td>
    <td>45.80</td>
    <td>84.35</td>
    <td>40.93</td>
    <td>42.90</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>11.44</td>
    <td>27.04</td>
    <td>85.01</td>
    <td>35.10</td>
    <td>52.36</td>
    <td>29.95</td>
    <td>42.81</td>
    <td>83.05</td>
    <td>39.87</td>
    <td><b>43.22</b></td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>13.84</b></td>
    <td><b>29.61</b></td>
    <td><b>86.18</b></td>
    <td><b>40.41</b></td>
    <td><b>57.30</b></td>
    <td><b>33.19</b></td>
    <td><b>47.59</b></td>
    <td>84.66</td>
    <td><b>41.77</b></td>
    <td>42.80</td>
  </tr>
  <tr>
    <th colspan="11">College of Science (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>3.50</td>
    <td>10.30</td>
    <td>82.33</td>
    <td>6.20</td>
    <td>43.32</td>
    <td>13.45</td>
    <td>33.89</td>
    <td><b>82.70</b></td>
    <td>37.26</td>
    <td>34.36</td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td><b>11.07</b></td>
    <td>20.96</td>
    <td>82.91</td>
    <td>8.57</td>
    <td>46.05</td>
    <td>21.45</td>
    <td>37.15</td>
    <td>79.90</td>
    <td>38.61</td>
    <td>33.65</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>8.22</td>
    <td>20.56</td>
    <td>82.85</td>
    <td>8.47</td>
    <td>45.58</td>
    <td>20.59</td>
    <td>36.20</td>
    <td>80.33</td>
    <td>38.74</td>
    <td>33.34</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>8.57</td>
    <td>21.19</td>
    <td>82.78</td>
    <td>8.49</td>
    <td>44.86</td>
    <td>20.48</td>
    <td>35.43</td>
    <td>80.18</td>
    <td>38.27</td>
    <td>33.14</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>8.92</td>
    <td>22.31</td>
    <td>82.92</td>
    <td>8.67</td>
    <td>46.03</td>
    <td>22.05</td>
    <td>37.22</td>
    <td>80.03</td>
    <td>39.17</td>
    <td>33.65</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td>8.93</td>
    <td>22.87</td>
    <td>82.94</td>
    <td>13.40</td>
    <td>46.13</td>
    <td>22.23</td>
    <td>37.91</td>
    <td>80.11</td>
    <td>39.96</td>
    <td>33.40</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>8.15</td>
    <td>20.56</td>
    <td>82.64</td>
    <td>8.46</td>
    <td>45.19</td>
    <td>20.81</td>
    <td>36.73</td>
    <td>80.85</td>
    <td>38.04</td>
    <td>33.95</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>9.86</td>
    <td><b>23.31</b></td>
    <td><b>83.51</b></td>
    <td><b>14.75</b></td>
    <td><b>48.62</b></td>
    <td><b>24.26</b></td>
    <td><b>40.18</b></td>
    <td>80.87</td>
    <td><b>40.60</b></td>
    <td>33.44</td>
  </tr>
  <tr>
    <th colspan="11">Pamplin College of Business (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>4.13</td>
    <td>12.78</td>
    <td>84.27</td>
    <td>28.00</td>
    <td>47.04</td>
    <td>17.90</td>
    <td>36.78</td>
    <td>85.38</td>
    <td>38.38</td>
    <td><b>23.97</b></td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>16.03</td>
    <td>29.67</td>
    <td>85.22</td>
    <td>43.33</td>
    <td>52.17</td>
    <td>29.32</td>
    <td>43.09</td>
    <td>83.56</td>
    <td>42.96</td>
    <td>21.43</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>15.49</td>
    <td>23.55</td>
    <td>84.11</td>
    <td>34.22</td>
    <td>47.53</td>
    <td>23.35</td>
    <td>37.67</td>
    <td>82.60</td>
    <td>39.15</td>
    <td>23.04</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>15.29</td>
    <td>26.85</td>
    <td>85.41</td>
    <td>41.07</td>
    <td>53.11</td>
    <td>27.92</td>
    <td>43.90</td>
    <td>83.48</td>
    <td>41.07</td>
    <td>21.62</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>15.82</td>
    <td>28.69</td>
    <td>85.71</td>
    <td>42.39</td>
    <td>54.02</td>
    <td>28.85</td>
    <td>44.81</td>
    <td>84.14</td>
    <td>42.07</td>
    <td>21.33</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td><b>16.09</b></td>
    <td>29.38</td>
    <td>85.90</td>
    <td>43.39</td>
    <td>54.78</td>
    <td>28.86</td>
    <td>44.58</td>
    <td>84.08</td>
    <td>42.92</td>
    <td>21.84</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>13.10</td>
    <td>22.31</td>
    <td>84.57</td>
    <td>35.42</td>
    <td>53.30</td>
    <td>23.70</td>
    <td>42.51</td>
    <td>84.43</td>
    <td>40.31</td>
    <td>23.62</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td><b>15.76</b></td>
    <td><b>32.86</b></td>
    <td><b>86.51</b></td>
    <td><b>46.58</b></td>
    <td><b>56.90</b></td>
    <td><b>34.50</b></td>
    <td><b>48.36</b></td>
    <td><b>85.40</b></td>
    <td><b>45.08</b></td>
    <td>22.53</td>
  </tr>
  <tr>
    <th colspan="11">Virginia Maryland College of Veterinary Medicine (VTechAGP)</th>
  </tr>
  <tr>
    <td>Pre-trained</td>
    <td>3.57</td>
    <td>11.12</td>
    <td>83.49</td>
    <td>22.99</td>
    <td>46.25</td>
    <td>15.08</td>
    <td>34.88</td>
    <td><b>84.83</b></td>
    <td>36.98</td>
    <td><b>43.12</b></td>
  </tr>
  <tr>
    <td>Multi-LoRAs</td>
    <td>9.80</td>
    <td>24.33</td>
    <td>83.99</td>
    <td>31.49</td>
    <td>49.37</td>
    <td>22.87</td>
    <td>37.83</td>
    <td>82.68</td>
    <td>39.81</td>
    <td>33.14</td>
  </tr>
  <tr>
    <td>Single LoRA</td>
    <td>9.32</td>
    <td>21.78</td>
    <td>84.37</td>
    <td>31.32</td>
    <td>48.60</td>
    <td>22.02</td>
    <td>37.56</td>
    <td>83.58</td>
    <td>41.13</td>
    <td>42.31</td>
  </tr>
  <tr>
    <td>AWG<sub>Random</sub></td>
    <td>10.92</td>
    <td>22.38</td>
    <td>84.20</td>
    <td>33.06</td>
    <td>47.25</td>
    <td>22.47</td>
    <td>35.17</td>
    <td>81.91</td>
    <td>36.61</td>
    <td>32.53</td>
  </tr>
  <tr>
    <td>AWG<sub>K-Means</sub></td>
    <td>11.01</td>
    <td>29.11</td>
    <td>85.11</td>
    <td><b>39.98</b></td>
    <td>52.65</td>
    <td>27.69</td>
    <td>42.22</td>
    <td>84.35</td>
    <td>42.17</td>
    <td>33.24</td>
  </tr>
  <tr>
    <td>AWG<sub>Contrastive</sub></td>
    <td><b>11.81</b></td>
    <td>29.26</td>
    <td>85.48</td>
    <td>39.32</td>
    <td>52.95</td>
    <td>27.97</td>
    <td>42.78</td>
    <td>84.58</td>
    <td>42.20</td>
    <td>33.94</td>
  </tr>
  <tr>
    <td>w/o Fusion</td>
    <td>9.80</td>
    <td>22.80</td>
    <td>84.00</td>
    <td>28.83</td>
    <td>48.16</td>
    <td>22.76</td>
    <td>39.58</td>
    <td>81.48</td>
    <td>40.26</td>
    <td>33.95</td>
  </tr>
  <tr>
    <td>Sci-LoRA</td>
    <td>11.32</td>
    <td><b>29.55</b></td>
    <td><b>85.90</b></td>
    <td>39.09</td>
    <td><b>53.80</b></td>
    <td><b>28.24</b></td>
    <td><b>44.00</b></td>
    <td>84.67</td>
    <td><b>42.45</b></td>
    <td>34.12</td>
  </tr>
</table>

Metareview score: 3.5
Metareview: According to the authors, prior lay paraphrasing models (e.g. for simplifying scientific and technical contents for lay audience) generally focus on one specific scientific domain or the other. The current paper utilizes mixture of LoRa adapters to dynamically mix adapters based on the domain of the text.
Summary of reasons to publish: * Overall, the paper presents a reasonable application of LoRA for multi-domain lay paraphrasing.
* The method shows strong performance boosts across different metrics compared to existing models and ablations.
 * Proposes a method to balance generalist LoRA features vs specialist ones.