HFT: Half Fine-Tuning for Large Language Models
Authors: Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun, Hua Wu
Accepted at: acl25

HFT: Half Fine-Tuning for Large Language Models

Tingfeng Hui1*, Zhenyu Zhang2†, Shuohuan Wang2, Weiran Xu1, Yu Sun2, Hua Wu2

1Beijing University of Posts and Telecommunications, Beijing, China
2Baidu Inc., Beijing, China
(huitingfeng,xuweiran)@bupt.edu.cn
(zhangzhenyu07,wangshuohuan)@baidu.com

Abstract
Large language models (LLMs) with one or more fine-tuning phases have become necessary to unlock various capabilities, enabling LLMs to follow natural language instructions and align with human preferences. However, it carries the risk of catastrophic forgetting during sequential training, the parametric knowledge or the ability learned in previous stages may be overwhelmed by incoming training data. This paper finds that LLMs can restore some original knowledge by regularly resetting partial parameters. Inspired by this, we introduce Half Fine-Tuning (HFT) for LLMs, as a substitute for full fine-tuning (FFT), to mitigate the forgetting issues, where half of the parameters are selected to learn new tasks. In contrast, the other half are frozen to retain previous knowledge. We provide a feasibility analysis from the optimization perspective and interpret the parameter selection operation as a regularization term. HFT could be seamlessly integrated into existing fine-tuning frameworks without changing the model architecture. Extensive experiments and analysis on supervised fine-tuning, direct preference optimization, and continual learning consistently demonstrate the effectiveness, robustness, and efficiency of HFT. Compared with FFT, HFT not only significantly alleviates the forgetting problem, but also achieves the best performance in a series of downstream benchmarks, with an approximately 30% reduction in training time.

1 Introduction
Large language models (LLMs) bring immense revolutions to various natural language processing applications with powerful language understanding and generation capabilities. Unsupervised large-scale pre-training for learning basic world knowledge (hereinafter referred to as basic knowledge), followed by one or more fine-tuning phases with supervised data or human feedback, is becoming a new training paradigm in the era of LLMs (Achiam et al., 2023; Touvron et al., 2023; Yang et al., 2025). As the fine-tuning phase proceeds, the enormous potential of LLMs is gradually unleashed to handle various downstream tasks, while the parametric knowledge previously learned and stored in the pre-trained model might face a considerable risk of catastrophic forgetting (Lin et al., 2024; Neeman et al., 2023; Dong et al., 2024). To maintain intrinsic basic knowledge, the most straightforward idea is to keep the pre-trained parameters unchanged and include extra modules (e.g., LoRAs or adapters) for learning task-specific abilities (Dou et al., 2023; Wu et al., 2024a). However, such architectural modifications pose significant obstacles to model deployment and continual fine-tuning.

Without changing model architecture, full fine-tuning (FFT) methods update all parameters to improve the performance of downstream tasks (Zhang et al., 2023c; Hui et al., 2025), in which the element-wise parameter difference between fine-tuned and pre-trained models (i.e., task vector) represents the knowledge shift during fine-tuning (Ilharco et al., 2023). Herein, a desirable task vector is expected to keep basic knowledge of pre-trained models and learn new specialized knowledge. Interestingly, recent work shows that partial dropping or trimming of the task vector has only milder impacts on target task (Yadav et al., 2023; Yu et al., 2023). In other words, partial new parameters are sufficient for the learning of new abilities, so the upcoming question is, is it possible that a portion of old parameters could maintain the capabilities of the pre-trained model?

To answer this question, we start with LLAMA 2-7B and LLAMA 2-CHAT-7B, and attempt to reset partial parameters of the chat-model to the pre-trained model, then prob the general abilities and basic knowledge of these models (see Figure 1). As
Figure 1: Performance of LLAMA 2-7B, LLAMA 2-CHAT-7B, and the Half-Reset model on six general abilities and three basic knowledge benchmarks. It is interesting that simply resetting half of the parameters of the chat-model to the pre-trained model could roughly restore a significant amount of forgotten basic knowledge while maintaining high-level general abilities performance.

a representative general-purpose fine-tuning practice, there is some improvement in the general abilities of LLAMA 2-CHAT-7B, while the basic knowledge falls off a cliff. It is consistent with previous observations, indicating the destruction of parametric knowledge stored in LLAMA 2-7B (Dou et al., 2023). To balance the emerging general abilities and the inherent basic knowledge, we intuitively select and reset half of the parameters1 of LLAMA 2-CHAT-7B and are pleasantly surprised to find that the Half-Reset model greatly resumes the basic knowledge in LLAMA 2-7B while remaining the excellent general abilities of LLAMA 2-CHAT-7B (More details in Section 2).

Inspired by these above observations, we propose Half Fine-Tuning (HFT), a simple yet effective approach for the training of LLMs and further extrapolate it to the continual fine-tuning scenarios. Specifically, in each round of fine-tuning, we randomly select and freeze half of the parameters, and only update the other half. This enables the model to retain the capabilities at the starting point while learning downstream tasks and maintain the best balance between previous abilities and new skills. Note that HFT does not change the model architecture or traditional fine-tuning paradigm, thus theoretically it can be applied to any setting where the standard full fine-tuning is previously applicable, including but not limited to supervised fine-tuning (SFT), direct preference optimization (DPO), continual learning (CL), etc.

To evaluate the effectiveness of HFT in instruction fine-tuning settings, we conduct extensive experiments with TÜLU V2 (Ivison et al., 2023) for SFT and UltraFeedback (Cui et al., 2023) for DPO. Simultaneously, we also extend experiments on TRACE (Wang et al., 2023a) for CL (i.e. multi-round fine-tuning) to validate the proposed method in a more extreme scenario. Experimental results demonstrate that HFT not only exhibits excellent talent in alleviating catastrophic forgetting but also achieves comparable or even better performance in learning new abilities compared to FFT. Further analysis reveals that regardless of which half (or even only about half) of the parameters are selected, HFT is capable of attaining tolerable performance gains and impressive efficiency improvements, which brings considerable competition to the routine fine-tuning paradigm. In summary, the main contributions of this paper are as follows:

(1) We reveal that by resetting half of the fine-tuned parameters to the startup state, it is possible to preliminary restore the primeval ability while maintaining new learning ability, which poses new opportunities to alleviate catastrophic forgetting and obtain an all-around LLM.

(2) We propose Half Fine-Tuning (HFT), which entails freezing half of the parameters while training the other half. It allows LLMs to acquire new abilities while retaining and utilizing previously learned knowledge in various training settings.

(3) Extensive experiments and analyses demonstrate the effectiveness and efficiency of HFT. Without any alterations to the model architecture, HFT, as a plug-and-play solution with only a few lines of code, exhibits the potential to supersede FFT in the era of LLMs.

1Here, we keep the embedding and lm_head layers unchanged as LLAMA 2-CHAT-7B, and select 50% of the parameters in transformer layers. The parameter ratios in this paper all follow this statistical calibre.
Figure 2: The schematic procedure of HFT with LLAMA 2’s architecture. In each stage, we selectively freeze half of the parameters at the category-level and update the other half. Best viewed in color.

2 Pilot Experiments

Considering that the partial task vector is capable of maintaining new abilities (Yadav et al., 2023; Yu et al., 2023), we attempt to roll back the primeval abilities of pre-trained models by resetting the remaining part of the task vector, thereby alleviating the catastrophic forgetting problem caused by fine-tuning. In this section, We employ the representative well-aligned LLM, LLAMA 2-CHAT-7B, and the corresponding pre-trained backbone, LLAMA 2-7B, as models for analysis.

Setup. To balance the original abilities and the enhanced capabilities gained through instruction tuning, we simply choose to reset 50% of the parameters in LLAMA 2-CHAT-7B to LLAMA 2-7B, so that half of the parameters are hoped to align with the new tasks, while the other half is intended to restore the old capabilities. In the implementation, we randomly select half of each transformer layer according to the category of the parameter matrix. Specifically, we choose two from four self-attention matrices (i.e., \( W_Q, W_K, W_V, W_O \)), and for the odd parameter number in LLAMA’s feed-forward layers (i.e., \( W_{up}, W_{down}, W_{gate} \)), we randomly select half of the transformer layers to choose two matrices and the other half to choose one. Such a fine-grained selection strategy ensures that the Half-Reset operation rolls back exactly 50% of the parameters.

To assess the performance of the pre-trained, chat, and half-reset models on both new and old capabilities, we follow (Ivison et al., 2023) and (Dou et al., 2023) to introduce two categories of evaluation benchmarks: (1) General Abilities, including MMLU, GSM8K, BBH, TyDiQA, TruthfulQA, and HumanEval, which measure the LLMs’ newly enhanced abilities to perform specific downstream tasks like examination, reasoning, and coding. (2) Basic Knowledge, including NaturalQuestion, TriviaQA, and HotpotQA, which reflect the parametric world knowledge in the pre-trained model and could be used to evaluate retention of the primeval capabilities. For more details about the datasets and evaluation metrics, please refer to Appendix A.3.1 and A.3.2

Results. From Figure 1, it is intuitive to observe significant improvement of LLAMA 2-CHAT-7B on several general ability benchmarks, as well as the comprehensive decline on the basic knowledge benchmarks. When selectively restoring half parameters to the pre-trained LLAMA 2-7B model, although there is a slight performance loss in the overall performance of general abilities, we witness the remarkable recovery of basic knowledge. In Appendix A.4.2, we attempt other possible half-reset solutions and provide more numerical results, all of which exhibit similar phenomena.

In conclusion, the pilot experiments demonstrate that (1) full parameter fine-tuning with large-scale instruction data disrupts the basic knowledge stored within pre-trained LLMs. (2) Through a simple half-reset operation, it is possible to restore the forgotten knowledge partially. Take another step forward, these findings open a new door for model merging, inspiring us to preserve some mastered abilities of the startup point by freezing partial parameters during fine-tuning.
3 Methodology

Without loss of generality, we consider a sequential (continual) learning setting with multiple tasks \( \mathcal{T} \), in which each task corresponds to a set of input-output pairs \( \mathcal{D}^t = \{ x_n^t, y_n^t \}_{n=1}^{N^t} \). In the training process, a single model aligns all the tasks sequentially, with only access to the specific dataset \( \mathcal{D}^t \) at \( t \)-th round. Formerly, given an LLM parameterized by \( \theta \), the entire process aims to optimize the following objective, which encompasses all the tasks,

\[
\mathcal{J}(\theta) = \max_{\theta} \sum_{t \in \{1, |\mathcal{T}|\}} \sum_{(x_n^t, y_n^t) \in \mathcal{D}^t} \log \mathbf{P}_{\theta^t}(y_n^t|x_n^t),
\]

where \( \log \mathbf{P}(\cdot) \) represents the probability distribution of the model’s output. When there is only one task, the learning process degenerates into the standard supervised fine-tuning (SFT) form.

Half Fine-Tuning. Next, we accordingly propose Half Fine-Tuning (HFT) to learn the upcoming new task while maintaining and utilizing old abilities. Figure 2 illustrates the overall workflow of HFT, regarding the intermediate repetitive transformer layers, we divide each layer into three blocks: self-attention, feed-forward, and layernorm, so as half of each block is selected for updating in this round, while the remaining half is frozen. Note that the frozen and updated parameters vary among each training round. In this way, HFT is more conducive to maintaining relative knowledge parity across different rounds during the sequential alignment process, thus exhibiting significant scalability in successive training. From the formula perspective, we define the parameters that remain unchanged during the \( t \)-th round as \( \psi^t \), and correspondingly, the parameters that align to the upcoming tasks as \( \vartheta^t \) (i.e., \( \theta^t = \{ \vartheta^t, \psi^t \} \)). The training objective in Equation 1 thus changes to

\[
\begin{aligned}
\mathcal{J}(\theta) &= \max_{\theta} \sum_{t \in \{1, |\mathcal{T}|\}} \sum_{(x_n^t, y_n^t) \in \mathcal{D}^t} \log \mathbf{P}_{\{\vartheta^t, \psi^t\}}(y_n^t|x_n^t), \\
&\quad s.t.\ \vartheta^t \leftarrow \vartheta^{t-1} - \eta \nabla_{\vartheta} \mathcal{L}(\vartheta^{t-1}) ,\ \psi^t \leftarrow \psi^{t-1},
\end{aligned}
\]

where \( \eta \) and \( \mathcal{L}(\cdot) \) represent the learning rate and loss function, \( \nabla_{\vartheta} \) indicates that we only consider the gradients of selected parameters in fine-tuning.

Why Half Fine-Tuning Works. Excluding heuristic motivations, we are also interested in the theoretical principles behind HFT. Theoretically, HFT could be regarded as exerting a parameter-level mask to vanilla FFT. In this part, we borrow the thread in (Fu et al., 2022) to interpret why HFT works from the perspective of optimization. Given a pre-trained model \( \mathcal{M}^0 \) with parameters \( \theta^0 \), the fine-tuned model \( \mathcal{M} \) with parameters \( \theta \) has the same structure as \( \mathcal{M}^0 \) such that \( \| \theta - \theta^0 \|_0 \leq p \dim(\theta) \), where \( p = 0.5 \) in HFT. Next, we denote \( M \in \{0, 1\}^{m \times m} \) as a mask diagonal matrix on the parameter, in which the diagonal is equal to 1 if the parameter is selected, thus the fine-tuning procedure can be formulated as \( \theta = \theta^0 + M \Delta \theta \), where \( \Delta \theta \) is the task vector. In that case, HFT solves an optimization problem with constraints \( \min_{\Delta \theta, M} \mathcal{L}(\theta^0 + M \Delta \theta) \) such that \( \| M \|_0 = \lfloor mp \rfloor; M_{ij} = 0,\ \forall i \neq j; M_{ii} \in \{0, 1\} \). where \( \mathcal{L} \) is the loss function, \( \lfloor \cdot \rfloor \) is the floor function, \( m \) is the parameter numbers. By integrating previous conditions, the optimization procedure of HFT can be reformulated as

\[
\mathcal{O} = \min_{\theta} \mathcal{L}(\theta)\ \ s.t.\ \| (I - M)(\theta - \theta^0) \|_2^2 = 0,
\]

With Lagrangian duality, solving the constrained optimization problem is equivalent to solving the following unconstrained optimization problem

\[
\mathcal{O}_L = \min_{\theta} \max_{\lambda} \mathcal{L}(\theta) + \lambda \| (I - M)(\theta - \theta^0) \|_2^2,
\]

where \( \lambda \) is the Lagrange multiplier. Based on the Minimax inequality, it is intuitive to derive that \( \min_{\theta} \max_{\lambda} \mathcal{L}(\theta) + \lambda \| (I - M)(\theta - \theta^0) \|_2^2 \geq \max_{\lambda} \min_{\theta} \mathcal{L}(\theta) + \lambda \| (I - M)(\theta - \theta^0) \|_2^2 \geq \min_{\theta} \mathcal{L}(\theta) + \| (I - M)(\theta - \theta^0) \|_2^2 \). In conclusion, the optimization process of HFT is equivalent to optimizing the upper bound of the FFT loss function \( \mathcal{L}(\theta) \) with a regularization term \( \| (I - M)(\theta - \theta^0) \|_2^2 \). From the optimization perspective, such regularization (with an appropriate sparsity \( M \)) contributes to the stability of the sparse fine-tuned model (Radya-Dixit and Wang, 2020; Fu et al., 2022), meaning that HFT has the opportunity to achieve results comparable to or even better than FFT.

4 Experiments

In this section, we primarily report the experimental results of full fine-tuning (FFT) and the proposed half fine-tuning (HFT) on supervised fine-tuning (with TÜLU V2 (Ivison et al., 2023) as training set), human preference alignment (with UltraFeedback (Cui et al., 2023)), and continual learning (with TRACE (Wang et al., 2023a)) scenarios, in which direct preference optimization (DPO) (Rafailov et al., 2023) is used to learn human preferences. Following (Ivison et al., 2023) and (Wang et al., 2023a), we employ LLAMA 2 and
<table>
  <tr>
    <th rowspan="2"> </th>
    <th colspan="2">MMLU (factuality)</th>
    <th colspan="2">GSMSK (reasoning)</th>
    <th colspan="2">BBH (reasoning)</th>
    <th colspan="2">TyDiQA (multilingual)</th>
    <th colspan="2">TruthfulQA (truthful)</th>
    <th colspan="2">HumanEval (coding)</th>
    <th rowspan="2">Overall</th>
  </tr>
  <tr>
    <th>EM (0-shot)</th>
    <th>EM (8-shot, CoT)</th>
    <th>EM (3-shot, CoT)</th>
    <th>F1 (1-shot, GP)</th>
    <th>MC2 (0-shot)</th>
    <th>Pass@10 (0-shot)</th>
  </tr>
  <tr>
    <th colspan="14"><i>Pre-trained models</i></th>
  </tr>
  <tr>
    <td>LLAMA 2-7B</td>
    <td>41.6</td>
    <td>12.0</td>
    <td>39.9</td>
    <td>48.4</td>
    <td>38.5</td>
    <td>26.2</td>
    <td>34.4</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B</td>
    <td>52.2</td>
    <td>34.5</td>
    <td>50.7</td>
    <td>50.3</td>
    <td>49.8</td>
    <td>32.7</td>
    <td>45.0</td>
  </tr>
  <tr>
    <th colspan="14"><i>Supervised Fine-Tuning (SFT) on TULU V2</i></th>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT</td>
    <td>48.5</td>
    <td>25.0</td>
    <td>42.2</td>
    <td>51.2</td>
    <td>41.7</td>
    <td><b>36.9</b></td>
    <td>41.0</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT (Reset)</td>
    <td>48.4</td>
    <td>23.0</td>
    <td>43.4</td>
    <td><b>52.4</b></td>
    <td>42.5</td>
    <td>32.5</td>
    <td>40.4</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT (Half)</td>
    <td><b>50.8</b></td>
    <td><b>30.5</b></td>
    <td><b>43.6</b></td>
    <td>52.3</td>
    <td><b>45.4</b></td>
    <td>34.6</td>
    <td><b>42.9</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT</td>
    <td>50.6</td>
    <td>45.0</td>
    <td>47.8</td>
    <td>55.0</td>
    <td>42.6</td>
    <td>42.4</td>
    <td>47.2</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT (Reset)</td>
    <td>52.7</td>
    <td>46.0</td>
    <td>52.8</td>
    <td>55.5</td>
    <td><b>46.8</b></td>
    <td>41.4</td>
    <td>49.2</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT (Half)</td>
    <td><b>54.5</b></td>
    <td><b>46.5</b></td>
    <td><b>53.7</b></td>
    <td><b>56.7</b></td>
    <td>45.7</td>
    <td><b>43.5</b></td>
    <td><b>50.1</b></td>
  </tr>
  <tr>
    <th colspan="14"><i>Direct Preference Optimization (DPO) on UltraFeedback</i></th>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO</td>
    <td>48.9</td>
    <td>28.0</td>
    <td>42.9</td>
    <td>50.2</td>
    <td><b>45.7</b></td>
    <td>35.6</td>
    <td><b>41.9</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO (Reset)</td>
    <td><b>49.0</b></td>
    <td><b>28.5</b></td>
    <td><b>43.1</b></td>
    <td>50.3</td>
    <td>43.3</td>
    <td>34.8</td>
    <td>41.5</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO (Half)</td>
    <td>48.8</td>
    <td>25.5</td>
    <td>42.8</td>
    <td><b>51.1</b></td>
    <td>45.5</td>
    <td><b>36.7</b></td>
    <td><b>41.7</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO</td>
    <td><b>52.0</b></td>
    <td>44.0</td>
    <td>47.1</td>
    <td>51.5</td>
    <td><b>45.5</b></td>
    <td><b>44.3</b></td>
    <td>47.4</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO (Reset)</td>
    <td>51.5</td>
    <td>46.5</td>
    <td>48.2</td>
    <td><b>53.7</b></td>
    <td>43.7</td>
    <td>42.7</td>
    <td>47.7</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO (Half)</td>
    <td>51.8</td>
    <td><b>48.5</b></td>
    <td><b>49.9</b></td>
    <td>52.9</td>
    <td>45.3</td>
    <td>41.0</td>
    <td><b>48.2</b></td>
  </tr>
</table>

Table 1: Results on general ability benchmarks of various models with instruction tuning (SFT, DPO), in which the default setting is FFT. Reset and Half refer to the proposed Half-Reset and Half Fine-Tuning methods, respectively. Bold text denotes the best result in each group.

LLAMA 2-CHAT as the backbone model, respectively. Appendix A.3 shows more information about implementations and Appendix A.4 proposes more additional experiments consisting of the impact of learning rates and random seeds, the exploration of DPO on HFT-based models, efficiency analysis and many other detailed results.

4.1 Experiments on Instruction Tuning

Setup. We employ the general abilities and basic knowledge benchmarks mentioned in Section 2 to evaluate various models under the instruction tuning settings. In Appendix 4.4, we introduce a series of sparse fine-tuning and model merging methods as additional baselines. To assess the conversation ability, we also compare these models on AlpacaEval 2.0 (see Appendix A.4.8).

Results on Improving General Abilities. Results in Table 1 demonstrate the effectiveness of our proposed HFT method, which simultaneously improves different specialized abilities by selectively fine-tuning half of the parameters. Specifically, compared to FFT under the SFT setting, HFT leads to an overall performance improvement of 1.9% on LLAMA 2-7B and 2.9% when scaling to LLAMA 2-13B. Furthermore, as we continue to perform DPO on SFT models, we observe that updating the policy model with HFT does not hinder the model from learning human preferences. In sum, the HFT method has strong robustness to adapt to different fine-tuning algorithms. Besides, we also review the Half-Reset method in Section 2, but the benefits of this approach are not robust, and we attribute it to the randomness of parameter operations. In comparison, HFT achieves a more stable performance improvement through the learning process, while avoiding the complexity of the two-stage process of fully updating followed by partially resetting.

Results on Preserving Basic Knowledge. When it comes to basic knowledge, as depicted in Table 2, both SFT and DPO exhibit a significant decline across all three benchmarks. Notably, HFT demonstrates excellent talent in preserving basic knowledge, consistently outperforming fully updating parameters during SFT and DPO. For example, during the SFT stage, HFT achieves improvements of 3.4% and 2.9% with LLAMA 2-7B and LLAMA 2-13B compared to FFT, respectively. It is worth mentioning that Half-Reset also shows a stable performance in alleviating knowledge forgetting, which once again confirms the motivation to keep partial initial parameters unchanged.

Remark. HFT not only effectively preserves a certain degree of basic knowledge of the pre-trained model, but also utilizes this knowledge to achieve better learning of new abilities.

4.2 Experiments on Continual Learning

Setup. We evaluate the performance in the continual learning setting (with TRACE (Wang et al., 2023a)), using four representative approaches and attempt to replace FFT with HFT. (1) SeqFT: It is a standard for sequentially learning all parameters of downstream tasks. (2) GEM (Lopez-Paz and
Table 2: Results on basic knowledge benchmarks of various models with instruction tuning.

Ranzato, 2017): It leverages episode memories to avoid forgetting, but it consumes extra computation time like other regularization-based methods. (3) Replay: It is a common strategy, here we integrate alignment data from LIMA (Zhou et al., 2023a) into the replay memory and replaying 10% of historical data. (4) LoraSeqFT (Hu et al., 2022): It sequentially updates the low-rank matrices while keeping the backbone fixed. Note that the LoRA-based method modifies the model architecture and is not suitable for combination with HFT. Following (Wang et al., 2023a), we start with LLAMA 2-CHAT-7B/13B, adopt Overall Performance (OP) and Backward Transfer (BWT) as the evaluation metrics (Appendix A.3.2 details the calculation process). Besides, we also report the general abilities and basic knowledge of various models after the final round of learning (see Appendix A.4.5).

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="2">FFT</th>
    <th colspan="2">HFT</th>
  </tr>
  <tr>
    <th>OP</th>
    <th>BWT</th>
    <th>OP</th>
    <th>BWT</th>
  </tr>
  <tr>
    <td>LLAMA 2-CHAT-7B</td>
    <td>6.4</td>
    <td>-45.2%</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>LoraSeqFT</td>
    <td>45.7</td>
    <td>-10.2%</td>
    <td>51.3 (+5.6)</td>
    <td>-5.6% (+4.6)</td>
  </tr>
  <tr>
    <td>GEM</td>
    <td>48.2</td>
    <td>-7.9%</td>
    <td>50.2 (+2.0)</td>
    <td>-5.9% (+2.0)</td>
  </tr>
  <tr>
    <td>Replay</td>
    <td>54.3</td>
    <td>1.4%</td>
    <td>54.1 (-0.2)</td>
    <td>+2.1% (+0.7)</td>
  </tr>
  <tr>
    <td>LLAMA 2-CHAT-13B</td>
    <td>26.5</td>
    <td>-30.0%</td>
    <td>-</td>
    <td>-</td>
  </tr>
  <tr>
    <td>LoraSeqFT</td>
    <td>49.0</td>
    <td>-9.4%</td>
    <td>52.0 (+3.0)</td>
    <td>-8.5% (+0.9)</td>
  </tr>
  <tr>
    <td>GEM</td>
    <td>50.4</td>
    <td>-8.9%</td>
    <td>53.6 (+3.2)</td>
    <td>-6.1% (+2.8)</td>
  </tr>
  <tr>
    <td>Replay</td>
    <td>54.7</td>
    <td>-0.6%</td>
    <td>57.4 (+2.7)</td>
    <td>+1.6% (+2.2)</td>
  </tr>
</table>

Table 3: OP and BWT on TRACE with different strategies. OP measures the learning of new tasks and BWT measures the forgetting of old tasks.

Results. Table 3 shows that the three FFT approaches could all benefit from equipping HFT. Specifically, HFT brings performance improvements of 5.7% and 2.0% on the OP metric in the SeqFT and GEM settings, respectively. It also boosts the performance with 4.6%, 0.7%, and 2.0% on the BWT metric based on the LLAMA 2-CHAT-7B. When scaling the model to 13b, HFT could also achieve superior performances. Further, fine-tuning with full parameters often suffers from severe catastrophic forgetting in the 5-th round (see Appendix A.4.15), while HFT does not experience such a problem in any of the rounds, making the learning process more stable. Besides, LoraSeqFT exhibits notably suboptimal performance in this setting. We assume that the knowledge capacity of the LoRA parameter is quite limited, thus resulting in considerable forgetting during the process of sequential training. On the contrary, HFT is based on a full set of parameters and selects half of the parameters to be fine-tuned in each round, which has a stronger knowledge tolerance.

Remark. HFT is naturally suitable for scenarios with continual fine-tuning, and (almost all) methods with FFT can be further improved by assembling HFT, highlighting the plug-and-play feature.

<table>
  <tr>
    <th></th>
    <th>OP</th>
    <th>BWT</th>
  </tr>
  <tr>
    <td>SeqFT (FFT)</td>
    <td>45.7</td>
    <td>-10.2%</td>
  </tr>
  <tr>
    <td>SeqFT (Model-level HFT)</td>
    <td>46.9 (+1.2)</td>
    <td>-9.2% (+1.0%)</td>
  </tr>
  <tr>
    <td>SeqFT (Layer-level HFT)</td>
    <td>47.9 (+2.2)</td>
    <td>-8.3% (+1.9%)</td>
  </tr>
  <tr>
    <td>SeqFT (Category-level HFT)</td>
    <td><b>51.3 (+5.6)</b></td>
    <td><b>-5.6% (+4.6%)</b></td>
  </tr>
</table>

Table 4: Different strategies for selecting half of the parameters on TRACE.

4.3 Impact of Parameter Selection

HFT heuristically selects parameters to be tuned or frozen. We hope to reveal the impact of parameter selection from parameter radio and selection strategy, to discuss the universality of the methodology.

Impact of Trainable Parameter Ratio. Firstly, we traverse the radio of parameters to be fine-tuned at a granularity of ~10% and evaluate the impact in both single-round and multi-round fine-tuning scenarios. From Figure 3, we observe that most of the results with only updating partial parameters are superior to FFT, and the performance is quite satisfactory when the trainable parameter radio is around 50%. In SFT, the performance of basic knowledge shows a clear downward trend with the increase of parameter ratio, while the general abilities slowly rise, which allows updating half or less of the parameters to have good performance. Meanwhile, when selecting half of the parameters during continual learning, the model reaches a balance of abilities between each round of tasks, resulting in a
more robust training procedure and optimal performance. This observation again confirms the early conjecture about catastrophic forgetting, especially in continual learning, it is necessary to freeze a portion of parameters in each round to preserve the capabilities of the previous models. Not only that, we also find that fixing partial parameters can improve training efficiency (see Table 10), and HFT could shorten the training time by 30% in FFT.

Impact of Selection Strategy. Next, we consider other possible strategies for selecting half of the parameters: (1) Model-level. It arbitrarily chooses half the number of parameter matrices, which may prevent the parameter ratio from accurately reaching 50%. (2) Layer-level. It selects all parameters of a layer every other layer. (3) Category-level. It selects based on parameter categories, which is the default strategy used in this paper, and ensures the accurate selection of 50% of the parameters. Table 4 reports the results of performing HFT on TRACE with sequential fine-tuning (SeqFT). The first noteworthy phenomenon is that all three selection strategies outperform the standard FFT, which once again confirms the motivation that freezing some parameters helps balance the old and new abilities in continual fine-tuning. Moreover, the category-level selection wins the best performance, we attribute it to the fine-grained strategy that maximizes the interaction between updated and non-updated parameters. From the perspective of model merging, it minimizes the damage to ready-made capabilities when performing a 50% dropout on the task vector, thereby providing greater possibilities for learning new tasks based on existing knowledge.

Remark. HFT is robust and insensitive to parameter selection, and selecting approximately 50% of the parameters with a reasonable selection strategy could achieve acceptable improvements.

4.4 More Baselines of Instruction Tuning

We highlight that the motivation of HFT is to alleviate the catastrophic forgetting problem during fine-tuning without changing the model architecture, which distinguishes it from PEFT methods such as LoRA. Based on this, we also introduce three extra groups of methods to illustrate the effectiveness of HFT. Specifically, we compare four sparse fine-tuning methods, LoRA (Hu et al., 2022), QLoRA (Dettmers et al., 2023), AdaLoRA (Zhang et al., 2023b), P-Tuning (Liu et al., 2022), and Mixout (Lee et al., 2020), three model merging methods, Average merging, TIES merging (Yadav et al., 2023), and DARE (Yu et al., 2023) and three layer freezing method, AutoFreeze (Liu et al., 2021), SmartFRZ (Li et al., 2024), and LISA (Pan et al., 2024). The experimental results are shown in Table 5, demonstrating that the HFT method achieves the best trade-off in both general abilities and basic knowledge benchmarks. The sparse fine-tuning methods preserve more basic knowledge but suffer more performance degradation in the general abilities evaluation, which is consistent with the previous conclusion that LoRA learns less and forgets less (Biderman et al., 2024). On the other hand, the model merging methods, in general, also perform worse than HFT. Additionally, model merging methods require FFT training followed by task vector pruning, making them more complex and time-consuming due to the two-stage process. For the Other layer freezing method, finally all the parameters are updated. According to the experiments and analysis in Section 2, this still cannot achieve the optimal performance.

5 Discussion

In this section, we further discuss the parameter changes in the fine-tuning process to deepen the understanding of HFT. We review the influence of embedding and lm_head layers, and visualize the parameter variations during successive training.

Revisit the Embedding and LM_head Layers. HFT defaults to updating the embedding and lm_head layers. Here, we aim to explore the roles of these two layers. Specifically, we freeze them while maintaining the same selection strategy and report results in SFT and continual learning. Since freezing the embedding and lm_head layers slightly reduces trainable parameters, we also include two models with similar parameter ratios that only freeze the parameters in transformer layers, to mitigate the impact of parameter ratio. As shown in Table 6, freezing these two layers leads to a substantial decline in knowledge-intensive benchmarks, especially for QA-related tasks. Experimental results in Table 7 witness another phenomenon, where forgetting metric BWT significantly increases while the learning metric OP faces a cliff-like decrease. Detailed results in Appendix A.4.13 reveal that there is a substantial decline in the performance of ScienceQA. To this extent, a preliminary conjecture emerges that the embedding and lm_head store information are
(a) Performance of models trained on TÜLU V2.

(b) Performance of models trained on TRACE.

Figure 3: Performance concerning different trainable parameter ratios. The solid lines mark the performance of HFT with various ratios and the dashed lines mark the FFT baseline.

<table>
  <tr>
    <th rowspan="2">MMLU</th>
    <th colspan="2">GSM 8K</th>
    <th colspan="2">BBH</th>
    <th colspan="2">TyDi QA</th>
    <th colspan="2">Truthful QA</th>
    <th colspan="2">Human Eval</th>
    <th colspan="2">Natural Question</th>
    <th colspan="2">Trivia QA</th>
    <th colspan="2">Hotpot QA</th>
    <th colspan="2">Overall</th>
  </tr>
  <tr>
    <th>LoRA</th>
    <th>QLoRA</th>
    <th>AdaLoRA</th>
    <th>P-tuning</th>
    <th>Mixout</th>
    <th>TIES (P+S)</th>
    <th>DARE (P+S)</th>
    <th>TIES (S+D)</th>
    <th>DARE (S+D)</th>
    <th>Average (S+D)</th>
    <th>AutoFreeze</th>
    <th>SmartFRZ</th>
    <th>LISA</th>
    <th>HFT (S)</th>
    <th>HFT (S)</th>
    <th>HFT (S)</th>
    <th>HFT (S)</th>
    <th>HFT (S)</th>
  </tr>
  <tr>
    <td colspan="19"><i>Sparse Fine-tuning Baselines</i></td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>46.8</td>
    <td>18.0</td>
    <td>39.5</td>
    <td>51.7</td>
    <td>44.8</td>
    <td>27.3</td>
    <td>12.7</td>
    <td>36.2</td>
    <td><b>17.8</b></td>
    <td>32.8</td>
  </tr>
  <tr>
    <td>QLoRA</td>
    <td>38.0</td>
    <td>2.5</td>
    <td>37.2</td>
    <td>15.0</td>
    <td>40.6</td>
    <td>24.0</td>
    <td>12.7</td>
    <td><b>43.2</b></td>
    <td>15.5</td>
    <td>25.4</td>
  </tr>
  <tr>
    <td>AdaLoRA</td>
    <td>47.2</td>
    <td>19.5</td>
    <td>39.1</td>
    <td>51.9</td>
    <td>44.4</td>
    <td>30.2</td>
    <td>12.3</td>
    <td>37.5</td>
    <td>16.9</td>
    <td>33.2</td>
  </tr>
  <tr>
    <td>P-tuning</td>
    <td>44.7</td>
    <td>16.5</td>
    <td>36.9</td>
    <td>50.2</td>
    <td>43.6</td>
    <td>26.5</td>
    <td>12.8</td>
    <td>40.9</td>
    <td>17.3</td>
    <td>32.2</td>
  </tr>
  <tr>
    <td>Mixout</td>
    <td>48.1</td>
    <td>24.5</td>
    <td>41.0</td>
    <td>49.8</td>
    <td>42.3</td>
    <td>33.7</td>
    <td>4.5</td>
    <td>28.2</td>
    <td>15.5</td>
    <td>32.0</td>
  </tr>
  <tr>
    <td colspan="19"><i>Model Merging Baselines</i></td>
  </tr>
  <tr>
    <td>TIES (P+S)</td>
    <td>47.8</td>
    <td>25.5</td>
    <td>40.2</td>
    <td>50.1</td>
    <td>43.3</td>
    <td>30.2</td>
    <td>5.5</td>
    <td>31.7</td>
    <td>14.4</td>
    <td>32.1</td>
  </tr>
  <tr>
    <td>DARE (P+S)</td>
    <td>49.2</td>
    <td>28.5</td>
    <td>42.9</td>
    <td><b>53.0</b></td>
    <td>44.4</td>
    <td>32.8</td>
    <td>6.1</td>
    <td>30.7</td>
    <td>15.1</td>
    <td>33.6</td>
  </tr>
  <tr>
    <td>TIES (S+D)</td>
    <td>39.6</td>
    <td>1.5</td>
    <td>39.7</td>
    <td>16.1</td>
    <td>38.4</td>
    <td>23.3</td>
    <td><b>12.9</b></td>
    <td>40.2</td>
    <td>15.6</td>
    <td>25.3</td>
  </tr>
  <tr>
    <td>DARE (S+D)</td>
    <td>45.8</td>
    <td>16.5</td>
    <td>40.4</td>
    <td>50.0</td>
    <td>42.7</td>
    <td>27.6</td>
    <td>5.8</td>
    <td>32.7</td>
    <td>14.1</td>
    <td>30.6</td>
  </tr>
  <tr>
    <td>Average (S+D)</td>
    <td>49.0</td>
    <td>22.0</td>
    <td><b>45.1</b></td>
    <td>52.8</td>
    <td>42.5</td>
    <td>32.6</td>
    <td>7.5</td>
    <td>35.6</td>
    <td>14.0</td>
    <td>33.5</td>
  </tr>
  <tr>
    <td colspan="19"><i>Layer Freezing Baselines</i></td>
  </tr>
  <tr>
    <td>AutoFreeze</td>
    <td>48.5</td>
    <td>25.5</td>
    <td>44.2</td>
    <td>50.1</td>
    <td>44.4</td>
    <td>28.3</td>
    <td>3.7</td>
    <td>30.2</td>
    <td>14.4</td>
    <td>32.1</td>
  </tr>
  <tr>
    <td>SmartFRZ</td>
    <td>46.7</td>
    <td>24.5</td>
    <td>43.7</td>
    <td>50.6</td>
    <td>43.8</td>
    <td>29.4</td>
    <td>4.5</td>
    <td>29.5</td>
    <td>13.8</td>
    <td>31.8</td>
  </tr>
  <tr>
    <td>LISA</td>
    <td>50.1</td>
    <td>27.0</td>
    <td>43.2</td>
    <td>51.7</td>
    <td>45.2</td>
    <td>29.7</td>
    <td>6.0</td>
    <td>31.0</td>
    <td>14.7</td>
    <td>33.2</td>
  </tr>
  <tr>
    <td>HFT (S)</td>
    <td><b>50.8</b></td>
    <td><b>30.5</b></td>
    <td><b>43.6</b></td>
    <td><b>52.3</b></td>
    <td><b>45.4</b></td>
    <td><b>34.6</b></td>
    <td><b>6.2</b></td>
    <td><b>32.8</b></td>
    <td><b>15.4</b></td>
    <td><b>34.6</b></td>
  </tr>
</table>

Table 5: General abilities and basic knowledge performance of more baselines. In model merging baselines, P, S and D refer to Pre-trained, SFT and DPO models, respectively.

highly relevant to world knowledge, so it is crucial to update them during the fine-tuning process.

Parameters Variation Analysis. To intuitively perceive the difference in model parameters between HFT and FFT, we visualize parameter variations of fine-tuned models relative to the initial model (LLAMA 2-CHAT-7B) during continual learning on TRACE. On the one hand, we group two adjacent layers and calculate the average variation of self-attention and feed-forward blocks, where average variation refers to the average of all matrix differences in the block of two models. On the other hand, based on the selected number of times in these eight rounds of fine-tuning, we compare the average variation of each block with FFT. Figure 4 shows variations from the perspective of the transformer block and selected time, respectively. Interestingly, we find that: (1) The parameter variation of each layer using HFT is fainter than those using FFT. (2) There is no significant difference in parameter variation between shallow and deep transformer layers, which is consistent in both fine-tuning settings. (3) The deviation from pre-trained parameters increases linearly with the time of selection, and the variations of parameters selected eight times are very similar to FFT. Therefore, the excessive offset of task vectors may not necessarily lead to an improvement in downstream performance but result in forgetting existing capabilities. <i>HFT seeks subtle balance by pulling back the task vector, alleviating catastrophic forgetting when learning subsequent tasks.</i>

6 Conclusion

In this paper, we observe that rolling back half of the parameters to the pre-trained state may re-
<table>
  <tr>
    <th></th>
    <th>MMLU</th>
    <th>GSM 8K</th>
    <th>BBH</th>
    <th>TyDi QA</th>
    <th>Truthful QA</th>
    <th>Human Eval</th>
    <th>Natural Questions</th>
    <th>Trivia QA</th>
    <th>Hotpot QA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>HFT<sub>38.9%</sub> (update E, H)</td>
    <td>49.9</td>
    <td>26.0</td>
    <td>44.6</td>
    <td>52.3</td>
    <td>45.0</td>
    <td>33.2</td>
    <td>6.3</td>
    <td>24.0</td>
    <td>14.1</td>
    <td>32.8</td>
  </tr>
  <tr>
    <td>HFT<sub>50.0%</sub> (update E, H)</td>
    <td>50.8</td>
    <td>30.5</td>
    <td>43.6</td>
    <td>52.3</td>
    <td>45.4</td>
    <td>34.6</td>
    <td>6.2</td>
    <td>32.8</td>
    <td>15.4</td>
    <td><b>34.6</b></td>
  </tr>
  <tr>
    <td>HFT<sub>61.1%</sub> (update E, H)</td>
    <td>49.0</td>
    <td>29.5</td>
    <td>42.7</td>
    <td>50.6</td>
    <td>49.6</td>
    <td>35.4</td>
    <td>6.6</td>
    <td>31.3</td>
    <td>16.1</td>
    <td>34.5</td>
  </tr>
  <tr>
    <td>HFT<sub>50.0%</sub> (freeze E, H)</td>
    <td>51.4</td>
    <td>29.0</td>
    <td>45.0</td>
    <td>50.5</td>
    <td>45.2</td>
    <td>35.0</td>
    <td>3.2</td>
    <td>24.1</td>
    <td>13.7</td>
    <td>33.0</td>
  </tr>
</table>

Table 6: General abilities and basic knowledge performance of HFT models fine-tuned on TÜLÜ V2 without embedding (E) and 1m_head (H) layers. Note that the subscript indicates the proportion of selected parameters of transformer layers.

![Radar charts showing parameter variations for SAN and FFN in various transformer blocks and selected times](page_320_370_1107_246.png)

(a) Variations on SAN in various transformer blocks. (b) Variations on FFN in various transformer blocks. (c) Variations on SAN with various selected times. (d) Variations on FFN with various selected times.

Figure 4: Parameters variations of the last round model fine-tuned on TRACE relative to the starting point LLAMA 2-CHAT-7B. The outer blue circle indicates FFT and the inner red circle indicates HFT.

<table>
  <tr>
    <th></th>
    <th>OP</th>
    <th>BWT</th>
  </tr>
  <tr>
    <td>HFT<sub>38.9%</sub> (update E, H)</td>
    <td>49.6</td>
    <td>-5.6%</td>
  </tr>
  <tr>
    <td>HFT<sub>50.0%</sub> (update E, H)</td>
    <td><b>51.3</b></td>
    <td>-5.6%</td>
  </tr>
  <tr>
    <td>HFT<sub>61.1%</sub> (update E, H)</td>
    <td>49.9</td>
    <td>-5.6%</td>
  </tr>
  <tr>
    <td>HFT<sub>50.0%</sub> (freeze E, H)</td>
    <td>46.1</td>
    <td><b>-2.2%</b></td>
  </tr>
</table>

Table 7: OP and BWT scores of HFT models fine-tuned on TRACE without embedding and 1m_head layers.

cover partial knowledge of the startup model while holding the performance of downstream tasks. Taking inspiration from this, we propose HFT, which adopts a category-level strategy to select half of the parameters for updating in each training round, and the remaining parameters are expected to maintain the learned knowledge. Extensive experiments on supervised fine-tuning, direct preference optimization, and continual learning scenarios demonstrate the effectiveness of HFT. It not only alleviates the catastrophic forgetting in preceding capabilities but also achieves comparable or even superior performance than FFT in downstream tasks. Further analysis shows that HFT is robust to selection strategies and selected parameter numbers. Moreover, HFT does not change the model architecture, making it easy to implement and scale, especially under successive fine-tuning scenarios.

Limitations

HFT achieves a balanced performance across general abilities and foundational knowledge benchmarks. It outperforms FFT while reducing training time by approximately 30%, and demonstrates strong scalability in continual fine-tuning scenarios. In contrast, widely adopted sparse fine-tuning methods such as LoRA underperform compared to HFT. Particularly in challenging settings like continual learning, these methods often fail and suffer from performance collapse. We believe HFT has the potential to succeed FFT as the preferred fine-tuning strategy in nearly all practical scenarios, owing to its superior performance and faster convergence. This work, however, has several limitations. Due to computational constraints, we focus on representative open-source models, such as LLAMA 2-7B and LLAMA 2-13B, without extending to larger models. Additionally, we evaluate HFT on standard dense architectures, leaving its application to structures such as MoE for future exploration. In the future, we aim to investigate HFT’s effectiveness across a broader range of model scales and architectures, while improving selection mechanisms to further boost performance.
References

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732.

Dan Biderman, Jose Gonzalez Ortiz, Jacob Portes, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham. 2024. Lora learns less and forgets less. Preprint, arXiv:2405.09673.

Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation.

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.

Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Liannmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See https://vicuna.lmsys.org (accessed 14 April 2023).

Jonathan H Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, pages 454–470.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168.

Mike Conover, Matt Hayes, Ankit Mathur, Xiangrui Meng, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, et al. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm.

OpenCompass Contributors. 2023. Opencompass: A universal evaluation platform for foundation models. https://github.com/open-compass/opencompass.

Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. Ultrafeedback: Boosting language models with high-quality feedback. arXiv preprint arXiv:2310.01377.

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36.

Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence, pages 220–235.

Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2024. How abilities in large language models are affected by supervised fine-tuning data composition. Preprint, arXiv:2310.05492.

Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Jun Zhao, Wei Shen, Yuhao Zhou, Zhiheng Xi, Xiao Wang, Xiaoran Fan, et al. 2023. Loramoe: Revolutionizing mixture of experts for maintaining world knowledge in language model alignment. arXiv preprint arXiv:2312.09979.

Zihao Fu, Haoran Yang, Anthony Man-Cho So, Wai Lam, Lidong Bing, and Nigel Collier. 2022. On the effectiveness of parameter-efficient fine-tuning. Preprint, arXiv:2211.15583.

Demi Guo, Alexander Rush, and Yoon Kim. 2021. Parameter-efficient transfer learning with diff pruning. In Annual Meeting of the Association for Computational Linguistics.

Mustafa B Gurbuz and Constantine Dovrolis. 2022. Nispa: Neuro-inspired stability-plasticity adaptation for continual learning in sparse networks. In International Conference on Machine Learning, pages 8157–8174.

Moonsu Han, Minki Kang, Hyunwoo Jung, and Sung Ju Hwang. 2019. Episodic memory reader: Learning what to remember for question answering from streaming data. arXiv preprint arXiv:1903.06164.

Zeyu Han, Chao Gao, Jinyang Liu, Sai Qian Zhang, et al. 2024. Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv preprint arXiv:2403.14608.

Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021a. Measuring massive multitask language understanding. In International Conference on Learning Representations.

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the math dataset. Preprint, arXiv:2103.03874.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR.

Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.

Tingfeng Hui, Pengyu Zhu, Bowen Ping, Ling Tang, Yaqi Zhang, and Sen Su. 2025. Decif: Improving instruction-following through meta-decomposition. Preprint, arXiv:2505.13909.

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations.

Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A Smith, Iz Beltagy, et al. 2023. Camels in a changing climate: Enhancing lm adaptation with tulu 2. arXiv preprint arXiv:2311.10702.

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. Preprint, arXiv:2403.07974.

James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, pages 3521–3526.

Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Richard Nagyfi, et al. 2023. Openassistant conversations-democratizing large language model alignment. Advances in Neural Information Processing Systems.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, pages 453–466.

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, Saumya Malik, Victoria Graf, Jena D. Hwang, Jiangjiang Yang, Ronan Le Bras, Oyvind Tafjord, Chris Wilhelm, Luca Soldaini, Noah A. Smith, Yizhong Wang, Pradeep Dasigi, and Hannaneh Hajishirzi. 2025. Tulu 3: Pushing frontiers in open language model post-training. Preprint, arXiv:2411.15124.

Cheolhyoung Lee, Kyunghyun Cho, and Wanmo Kang. 2020. Mixout: Effective regularization to finetune large-scale pretrained language models. Preprint, arXiv:1909.11299.

Sheng Li, Geng Yuan, Yue Dai, Youtao Zhang, Yanzhi Wang, and Xulong Tang. 2024. Smartfrz: An efficient training framework using attention-based layer freezing. Preprint, arXiv:2401.16720.

Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. 2019. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International conference on machine learning, pages 3925–3934. PMLR.

Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval.

Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. 2024. The unlocking spell on base LLMs: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations.

Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252. Association for Computational Linguistics.

Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxia Du, Zhilin Yang, and Jie Tang. 2022. Ptuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. Preprint, arXiv:2110.07602.

Yuhan Liu, Saurabh Agarwal, and Shivaram Venkataraman. 2021. Autofreeze: Automatically freezing model blocks to accelerate fine-tuning. Preprint, arXiv:2102.01386.

David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30.

Qijun Luo, Hengxu Yu, and Xiao Li. 2024. Badam: A memory efficient full parameter optimization method for large language models. Preprint, arXiv:2404.02827.

Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023. An empirical study
of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747.

Rabeeh Karimi Mahabadi, Sebastian Ruder, Mostafa Dehghani, and James Henderson. 2021. Parameter-efficient multi-task fine-tuning for transformers via shared hypernetworks. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 565–576.

Ella Neeman, Roee Aharoni, Or Honovich, Leshem Choshen, Idan Szpektor, and Omri Abend. 2023. Disentqa: Disentangling parametric and contextual knowledge with counterfactual question answering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10056–10070.

Rui Pan, Xiang Liu, Shizhe Diao, Renjie Pi, Jipeng Zhang, Chi Han, and Tong Zhang. 2024. Lisa: Layerwise importance sampling for memory-efficient large language model fine-tuning. Preprint, arXiv:2403.17919.

Bohao Peng, Zhuotao Tian, Shu Liu, Mingchang Yang, and Jiaya Jia. 2024. Scalable language model with generalized continual learning. arXiv preprint arXiv:2404.07470.

Qwen, .; An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115.

Evani Radiya-Dixit and Xin Wang. 2020. How fine can fine-tuning be? learning efficient language models. In International Conference on Artificial Intelligence and Statistics, pages 2435–2443. PMLR.

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems.

Anastasia Razdaiibiedina, Yunjing Mao, Rui Hou, Maidian Khabsa, Mike Lewis, and Amjad Almahairi. 2023. Progressive prompts: Continual learning for language models. arXiv preprint arXiv:2301.12314.

David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. Advances in Neural Information Processing Systems, 32.

Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. Challenging BIG-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003–13051. Association for Computational Linguistics.

Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashylykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence.

Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, et al. 2023a. Trace: A comprehensive benchmark for continual learning in large language models. arXiv preprint arXiv:2310.06762.

Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13484–13508. Association for Computational Linguistics.

Yizhong Wang, Swaroop Mishra, Pegah Alipoormalabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal Kumar Pal, Maitreyaa Patel, Mehrad Moradshahi, Mihir Parmar, Mirali Purohit, Neeraj Varshney, Phani Rohitha Kaza, Pulkit Verma, Ravsehaj Singh Puri, Rushang Karia, Savan Doshi, Shailaja Keyur Sampat, Siddhartha Mishra, Sujan Reddy A, Sumanta Patro, Tanay Dixit, and Xudong Shen. 2022. Super-NaturalInstructions: Generalization via declarative instructions on 1600+ NLP tasks. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5085–5109. Association for Computational Linguistics.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou,
et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, pages 24824–24837.

Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ping Luo, and Ying Shan. 2024a. Llama pro: Progressive llama with block expansion. arXiv preprint arXiv:2401.02415.

Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. 2024b. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364.

Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023a. Wizardlm: Empowering large language models to follow complex instructions. Preprint, arXiv:2304.12244.

Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023b. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196.

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. Ties-merging: Resolving interference when merging models. In Thirty-seventh Conference on Neural Information Processing Systems.

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaksi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xucheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.

Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600.

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li. 2023. Language models are super mario: Absorbing abilities from homologous models as a free lunch. arXiv preprint arXiv:2311.03099.

Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. 2021. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199.

Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023a. Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512.

Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karapantziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023b. Adalora: Adaptive budget allocation for parameter-efficient fine-tuning. Preprint, arXiv:2303.10512.

Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyu Hu, Tianwei Zhang, Fei Wu, et al. 2023c. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792.

Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023a. Lima: Less is more for alignment. Preprint, arXiv:2305.11206.

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023b. Instruction-following evaluation for large language models. Preprint, arXiv:2311.07911.

A Appendix

A.1 Discussion of Selecting Ratios
Yu et al. (2023) have found that parameters are redundant during the SFT process, and excellent performance can be achieved for downstream tasks without the need for full parameter fine-tuning. In section 2, we also find that there is a correspondence between parameters and abilities. From the perspective of forgetting and knowledge conflict, HFT helps mitigate catastrophic forgetting by maximizing the balance between existing capabilities and newly introduced abilities through selecting half of the parameters, thereby reducing conflicts between different knowledge and improving performance. Furthermore, in Section 3 of our paper, we provide a theoretical analysis demonstrating that HFT optimize the upper bound of the FFT loss function with a regularization term.

A.2 Related Work
Sparse Fine-Tuning. With the continuous increase in the number of language model parameters, sparse fine-tuning (a.k.a. parameter-efficient fine-tuning (PEFT)) offers an effective solution by reducing trainable parameters while achieving comparable performance to FFT (Fu et al., 2022; Ding et al., 2023; Han et al., 2024). Adapter (Houlsby
Algorithm 1: Algorithm of HFT with Category-Level Parameter Selection

Input: Pre-trained model \( \theta_0 \)
Initialize sequential training task \( \mathcal{T} \) with data \( \mathcal{D}_t \), feed-forward block container FFNs=[], self-attention block container SANs=[], and layernorm block container LNs=[].
for \( t = 1 \) to \( |\mathcal{T}| \) do
    // Set all parameters to retain gradients before each fine-tuning stage
    foreach param in \( \theta_{t-1} \) do
        param.requires_grad = True
    // Omit the embedding and lm_head layer
    mark_layers = random.sample(transformer_layers, len(transformer_layers)//2)
    foreach layer in transformer_layers do
        foreach param in layer do
            if param belongs to FFN block then
                FFNs.append(param)
            else if param belongs to SAN block then
                SANs.append(param)
            else
                LNs.append(param)
    // For FFNs with an odd number of parameters in one layer, the number of selected parameters in half of the layers is rounded up, while the other half is rounded down.
    if layer in mark_layers then
        freeze_ffn = random.sample(FFNs, [len(FFNs)/2])
    else
        freeze_ffn = random.sample(FFNs, [len(FFNs)/2])
    freeze_san = random.sample(SANs, len(SANs)//2)
    freeze_ln = random.sample(LNs, len(LNs)//2)
    foreach param in freeze_ffn, freeze_san and freeze_ln do
        param.requires_grad = False
    Set FFNs, SANs and LNs to []
Model training process on with dataset \( \mathcal{D}_t \)

Output: Fine-tuned model \( \theta_{|\mathcal{T}|} \)

et al., 2019; Mahabadi et al., 2021; Zhang et al., 2023a) and LoRA (Hu et al., 2022; Dou et al., 2023; Dettmers et al., 2023), the two most famous kinds of work, freeze the initial model weight and inject an adapter or a trainable rank decomposition matrices into each layer. However, these approaches change the model architecture and therefore require customized deployment. Keeping the architecture unchanged, DiffPruning (Guo et al., 2021) learns a sparse diff vector for each task, enabling PEFT to scale well with new tasks. BitFit (Zaken et al., 2021) only fine-tunes the bias terms of BERT and achieves considerably good performance. Unfortunately, these methods designed for specific tasks or networks (e.g., bias) are unsuitable for modern general-purpose large-scale models. From the perspective of low GPU memory overhead, BAdam (Luo et al., 2024) randomly divides the entire parameter into multiple blocks and updates each block sequentially, LISA (Pan et al., 2024) changes the granularity of blocks at the layer level. Besides, Mixout (Lee et al., 2020) resets a portion of neurons to a pre-trained state in each training step. In this way, all parameters in BAdam, LISA, and Mixout are updated, which is different from HFT and not conducive to continual learning.

Continual Learning. Continual learning aims to develop learning algorithms that can accumulate knowledge on non-stationary data, and vanilla FFT has been proven to lead to severe catastrophic forgetting issues when adapting to incoming streaming tasks (Luo et al., 2023; Wang et al., 2024). To address this issue, experience replay (Rolnick et al., 2019; Peng et al., 2024) is a widely used technique that incorporates a portion of data from previous rounds into the current training process. Regularization-based models (Kirkpatrick et al., 2017; Lopez-Paz and Ranzato, 2017) introduce additional terms in the loss function to penalize changes in crucial weights. Parameter-allocation approaches (Li et al., 2019; Gurbuz and Dovrolis, 2022) feature an isolated parameter subspace dedicated to each task throughout the network. When LLMs enter the era of billions of parameters, researchers prefer to use progressive prompts (Razdaibiedina et al., 2023) or PEFT (Dou et al., 2023; Wu et al., 2024a) to tune a powerful general backbone for specific tasks or domains (Wu et al.,
2024b). Instead of introducing auxiliary modules or losses, HFT explores a new direction based on the characteristics of LLMs, proving that random parameter selection is sufficient to achieve passable performance and has the potential to become a successor to FFT.

A.3 Experimental Setup

A.3.1 Datasets
To validate the performance of supervised fine-tuning, we choose TÜLU V2 (Ivison et al., 2023) which is a combination of high-quality open resources, including datasets (1) created by researchers from existing NLP datasets (e.g. SuperNI (Wang et al., 2022)), (2) written by humans (e.g. Dolly (Conover et al., 2023) and Open Assistant (Köpf et al., 2023)), (3) generated by LLMs (e.g. Self-Instruct (Wang et al., 2023b), Alpaca (Taori et al., 2023) and Baize (Xu et al., 2023b)), (4) comprised of user-shared prompts accompanied by model-generated completions (e.g. ShareGPT (Chiang et al., 2023)), and (5) developed for specific abilities (e.g. CoT (Wei et al., 2022) for chain-of-thought and Code-Alpaca (Chaudhary, 2023) for code generation). To examine the capacity for reinstating a fraction of impaired capabilities while adhering to human preferences, we utilize UltraFeedback (Cui et al., 2023) which is a large-scale, high-quality, and diversified preference dataset. For continual learning, we select TRACE (Wang et al., 2023a), a novel benchmark designed for continual learning (CL) in LLMs, to evaluate catastrophic forgetting in standard CL settings. TRACE consists of 8 distinct datasets spanning challenging tasks, domain-specific tasks, multilingual capabilities, code generation, and mathematical reasoning.

A.3.2 Evaluation Metrics
Supervised Fine-Tuning and Direct Preference Optimization. To validate the effectiveness of our method, we employ general abilities and basic knowledge benchmarks to assess the performance in learning new tasks and preserving the original capabilities, respectively. Specifically, for the general abilities benchmarks, we include the following evaluation sets to test various abilities. (1) Factual knowledge: To assess the LLMs’ factual knowledge, we employ the Massive Multitask Language Understanding dataset (MMLU) (Hendrycks et al., 2021a). MMLU comprises a collection of questions across 57 subjects from elementary to professional difficulty levels. We report the 5-shot accuracy based on answer perplexity. (2) Reasoning: We utilize the test split of the Grade School Math (GSM8K) dataset (Cobbe et al., 2021) and Big-Bench-Hard (BBH) (Suzgun et al., 2023) to evaluate the reasoning abilities. We report the 8-shot accuracy and the exact match (EM) rates for GSM8K and BBH, respectively. (3) Multilingualism: To evaluate multilingual capabilities, we employ TyDiQA (Clark et al., 2020), a multilingual question-answering benchmark that covers 11 typologically diverse languages. We adopt the gold-passage setup, where a passage containing the reference answer is provided, and report the F1 score. (4) Coding: To evaluate the LLMs’ ability to generate functionally correct programs from doc-strings, we utilize HumanEval (Chen et al., 2021) and report the pass@10 performance. (5) Truthful: We incorporate TruthfulQA (Lin et al., 2022) to assess the ability to avoid generating known falsehoods resulting from misconceptions or false beliefs while providing informative responses. (6) Conversation: We use AlpacaEval 2.0 (Li et al., 2023) to evaluate the instruction-following abilities. AlpacaEval is an LLM-based automatic evaluation metric. In this paper, we calculate the win rates against the GPT-4-preview-1106. We include the following three datasets for basic knowledge benchmarks to validate the basic knowledge preserved in LLMs: (1) NaturalQuestion (Kwiatkowski et al., 2019), (2) TriviaQA (Han et al., 2019), and (3) HotpotQA (Yang et al., 2018).

Continual Learning. For continual learning evaluations, following (Wang et al., 2023a), we use Overall Performance (OP) and Backward Transfer (BWT) scores as the main metrics in CL settings. In terms of the formula, after incrementally learning the \( t \)-th task, the performance on the \( i \)-th task (where \( i \leq t \)) is denoted as \( S_{t,i} \). The OP and BWT scores can be calculated as

\[
\text{OP}_t = \frac{1}{t} \sum_{i=1}^{t} S_{t,i}, \quad \text{BWT}_t = \frac{1}{t} \sum_{i=1}^{t-1} (S_{t,i} - S_{i,i}) .
\]

We utilize accuracy as the primary evaluation metric for C-STANCE, FOMC, ScienceQA, NumGLUE-cm, and NumGLUE-ds. In the case of Py150, we employ similarity as the evaluation metric. Moreover, for the evaluation of MeetingBank and 20Minuten, we employ the ROUGE-L metric.
<table>
  <tr>
    <th>MMLU</th>
    <th>GSM 8K</th>
    <th>BBH</th>
    <th>TyDi QA</th>
    <th>Truthful QA</th>
    <th>Human Eval</th>
    <th>Natural Question</th>
    <th>Trivia QA</th>
    <th>Hotpot QA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>59.4</td>
    <td>61.0</td>
    <td>59.2</td>
    <td>56.2</td>
    <td>50.1</td>
    <td>68.5</td>
    <td>5.1</td>
    <td>48.6</td>
    <td>20.7</td>
    <td>47.8</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>61.2</td>
    <td>63.5</td>
    <td>58.3</td>
    <td>60.4</td>
    <td>50.5</td>
    <td>67.9</td>
    <td>10.9</td>
    <td>55.5</td>
    <td>21.6</td>
    <td><b>50.0</b></td>
  </tr>
</table>

Table 8: General abilities and basic knowledge performance of LLAMA 3 8B.

<table>
  <tr>
    <th></th>
    <th>LLAMA 2-7B</th>
    <th>LLAMA 2-CHAT-7B</th>
    <th>Model-level Half-Reset</th>
    <th>Layer-level Half-Reset</th>
    <th>Category-level Half-Reset</th>
  </tr>
  <tr>
    <td>MMLU (EM, 0-shot)</td>
    <td>41.6</td>
    <td>47.0</td>
    <td>46.2</td>
    <td>45.8</td>
    <td>46.7</td>
  </tr>
  <tr>
    <td>GSM (ACC, 8-shot)</td>
    <td>12.0</td>
    <td><b>26.0</b></td>
    <td>8.0</td>
    <td>22.0</td>
    <td>24.0</td>
  </tr>
  <tr>
    <td>BBH (EM, 0-shot)</td>
    <td>39.9</td>
    <td>39.2</td>
    <td><b>41.0</b></td>
    <td>39.5</td>
    <td>37.7</td>
  </tr>
  <tr>
    <td>TyDiQA (F1, 1-shot)</td>
    <td><b>48.4</b></td>
    <td>43.6</td>
    <td><b>46.3</b></td>
    <td>44.2</td>
    <td>44.9</td>
  </tr>
  <tr>
    <td>TruthfulQA (MC2, 0-shot)</td>
    <td>38.5</td>
    <td><b>46.0</b></td>
    <td>41.7</td>
    <td><b>43.1</b></td>
    <td>41.7</td>
  </tr>
  <tr>
    <td>HumanEval (Pass@10)</td>
    <td>26.2</td>
    <td>23.9</td>
    <td><b>26.8</b></td>
    <td>25.0</td>
    <td>22.0</td>
  </tr>
  <tr>
    <th><i>Overall (General Ability)</i></th>
    <th>34.4</th>
    <th><b>37.6</b></th>
    <th>35.0</th>
    <th><b>36.6</b></th>
    <th><b>36.2</b></th>
  </tr>
  <tr>
    <td>NaturalQuestion (EM, 0-shot)</td>
    <td>12.9</td>
    <td>7.2</td>
    <td>8.2</td>
    <td><b>11.2</b></td>
    <td>10.9</td>
  </tr>
  <tr>
    <td>TriviaQA (EM, 0-shot)</td>
    <td><b>40.2</b></td>
    <td>3.3</td>
    <td>18.3</td>
    <td><b>21.3</b></td>
    <td><b>21.3</b></td>
  </tr>
  <tr>
    <td>HotpotQA (EM, 0-shot)</td>
    <td>15.6</td>
    <td>6.6</td>
    <td>7.4</td>
    <td>9.9</td>
    <td>9.0</td>
  </tr>
  <tr>
    <th><i>Overall (World Knowledge)</i></th>
    <th>22.9</th>
    <th>5.7</th>
    <th>11.3</th>
    <th>12.4</th>
    <th><b>13.7</b></th>
  </tr>
  <tr>
    <th><i>Overall</i></th>
    <th>30.6</th>
    <th>27.0</th>
    <th>27.1</th>
    <th>28.5</th>
    <th><b>28.7</b></th>
  </tr>
</table>

Table 9: General abilities and basic knowledge results of LLAMA 2-7B, the well-aligned model LLAMA 2-CHAT-7B, and our proposed three half-reset approaches.

A.3.3 Implementation Details

Following (Ivison et al., 2023), in the SFT phase on TÜLU V2, we adopt a linear-decreasing learning rate of 2e-5 with a 0.3 warmup ratio and train for 2 epochs. For the human preference alignment phase on UltraFeedback, we use direct preference optimization (Rafailov et al., 2023) to align the fine-tuned LLMs on TÜLU V2. We use a learning rate of 5e-7 and a global batch size of 32. Due to the context length of 4096 used during LLAMA 2 pre-training, as referenced in the (Ivison et al., 2023) code repository issues, we set a maximum sequence length of 4096 during the SFT stage. However, due to hardware resource limitations, the maximum sequence length is reduced to 1024 during the DPO stage under LLAMA 2-13B. During the continual learning phase, following (Wang et al., 2023a), we employ a fixed learning rate of 1e-5 and fine-tune the eight sub-datasets for different numbers of epochs: 5, 3, 7, 5, 3, 5, 5, and 7 epochs, respectively. The global batch size for both stages is set to 128. All our experiments are conducted on one machine equipped with 8x80G Nvidia A100. Algorithm 1 introduce the detailed implementations of our proposed fine-grained selecting approach of HFT. Additionally, to evaluate the SFT and DPO models, we employ a chat format, using specialized tokens <user/> and <assistant/> to mark user utterances and target assistant responses, respectively. However, we use a standard language format for HumanEval and the basic knowledge benchmarks when evaluating pre-trained models.

As for the implementation of HFT, our HFT parameter selection method primarily focuses on each transformer block. For the parameter matrices within each block, we first categorize them into three types: attention, MLP, and other. Specifically, for the LLAMA 2 model, in the attention category, each block contains four attention layers (Q, K, V, and O), and we randomly select two of these layers to freeze. In the "other" category, each block includes the input layer norm and post-attention layer norm, and we randomly select one of these layers to freeze. Finally, for the MLP category, each block contains up, down, and gate layers. Since there is an odd number of layers, to maintain a 50% parameter selection ratio, we freeze two layers in every other block, while freezing one layer in the remaining blocks.

A.4 Additional Experiments

A.4.1 More Trials on Other Transformer Architecture

We mainly focus on the performance of HFT on standard dense transformers and experiment with the most representative open-source models. Here,
<table>
  <tr>
    <th># Trainable Parameters (%)</th>
    <th>8.3</th>
    <th>22.3</th>
    <th>30.6</th>
    <th>38.9</th>
    <th>50.0</th>
    <th>61.1</th>
    <th>69.4</th>
    <th>77.7</th>
    <th>91.7</th>
    <th>100</th>
  </tr>
  <tr>
    <td>Runtime (%)</td>
    <td>48.0</td>
    <td>52.2</td>
    <td>56.4</td>
    <td>64.0</td>
    <td>68.5</td>
    <td>72.5</td>
    <td>85.1</td>
    <td>85.2</td>
    <td>89.0</td>
    <td>100</td>
  </tr>
  <tr>
    <td>\( \Delta (\%) \)</td>
    <td>-52.0</td>
    <td>-47.8</td>
    <td>-43.6</td>
    <td>-36.0</td>
    <td>-31.5</td>
    <td>-27.5</td>
    <td>-14.9</td>
    <td>-14.8</td>
    <td>-11.0</td>
    <td>0.0</td>
  </tr>
</table>

Table 10: Efficiency analysis among different ratios of trainable parameters, in which FFT as a reference value and underline marks HFT proposed in this paper.

<table>
  <tr>
    <th></th>
    <th>MMLU</th>
    <th>GSM 8K</th>
    <th>BHH</th>
    <th>TyDi QA</th>
    <th>Truthful QA</th>
    <th>Human Eval</th>
    <th>Natural Question</th>
    <th>Trivia QA</th>
    <th>Hotpot QA</th>
    <th>Overall</th>
  </tr>
  <tr>
    <td>DPO (FFT-based, 7b)</td>
    <td>48.8</td>
    <td>25.5</td>
    <td>42.8</td>
    <td>51.1</td>
    <td>45.5</td>
    <td>36.7</td>
    <td>1.9</td>
    <td>22.9</td>
    <td>12.8</td>
    <td>32.0</td>
  </tr>
  <tr>
    <td>DPO (HFT-based, 7b)</td>
    <td>50.7</td>
    <td>30.5</td>
    <td>42.8</td>
    <td>43.9</td>
    <td>49.8</td>
    <td>35.1</td>
    <td>1.0</td>
    <td>20.4</td>
    <td>5.9</td>
    <td>31.1</td>
  </tr>
  <tr>
    <td>DPO (FFT-based, 13b)</td>
    <td>51.8</td>
    <td>48.5</td>
    <td>49.9</td>
    <td>52.9</td>
    <td>45.3</td>
    <td>41.0</td>
    <td>0.2</td>
    <td>5.5</td>
    <td>3.0</td>
    <td>33.1</td>
  </tr>
  <tr>
    <td>DPO (HFT-based, 13b)</td>
    <td>55.0</td>
    <td>45.5</td>
    <td>51.4</td>
    <td>53.2</td>
    <td>49.5</td>
    <td>42.9</td>
    <td>0.3</td>
    <td>4.9</td>
    <td>4.7</td>
    <td>34.2</td>
  </tr>
</table>

Table 11: General abilities and basic knowledge performance of DPO stage (with HFT), which is initialized with HFT-based SFT models fine-tuned on TÜLU V2.

we conduct an additional experiment to compare HFT and FFT on LLAMA 3-8B, which uses group-query attention and differs from the multi-head attention in LLAMA 2. We make some modifications to the selection method of HFT when applied to GQA. Specifically, since each key-value \(( (K, V) )\) pair corresponds to multiple queries \(Q\), GQA maintains dimensional consistency in matrix operations by duplicating the \((K, V)\) pairs. On a macro level, in terms of matrix representation, the dimensions of \(K\) and \(V\) matrices are smaller than that of \(Q\). Therefore, we separate \(K\) and \(V\) as one group and \(Q\) as another for selection, rather than filtering \(Q, K, V\) together as traditionally done in MHA. As shown in Table 8, HFT still achieved the best performance, which also proves the universality and robustness of our method.

A.4.2 Detailed Results of Pilot Experiments

Table 9 presents the detailed results of pilot experiments conducted in Section 2. We also compare two additional model-level and layer-level parameter selection methods here. The results indicate that the category-level selection approach achieves the highest overall performance, consistent with the follow-up training setting conclusion.

A.4.3 Efficiency Analysis

We conduct a comparison of the runtime costs for different ratios of trainable parameters. Specifically, we fine-tuned LLAMA 2-7B on TÜLU V2 and record the total duration from the start to the end of the training program. The results in Table 10 demonstrate that, without specific optimization, all models with varying ratios of trainable parameters can reduce the training time. As expected, as the proportion of trainable parameters increases, the training duration also increases. Notably, our HFT method achieves a 31.5% reduction in training time, significantly decreasing the training cost for extremely large-scale instruction datasets.

A.4.4 Direct Preference Optimization with HFT-based Models

In Section 4.1, we initialize our DPO process with the FFT model. In this section, we investigate the performance of the DPO process when initialized with the HFT model. The experimental results are shown in Table 11. We observe that while the DPO process on the HFT model performs better in certain general abilities., it experiences minor losses in overall performance under LLAMA 2-7B. However, the situation is reversed in LLAMA 2-13B, where the DPO deployed on the HFT model outperforms the FFT-initialized DPO. Nonetheless, DPO equipped with HFT tends to improve performance compared to DPO with FFT consistently.

A.4.5 General Abilities and Basic Knowledge of Continual Fine-tuned Models

We also evaluate the models mentioned in Section 4.2 on general abilities and basic knowledge benchmarks. The experimental results are presented in Table 12. We observe that after 8 rounds of fine-tuning on consecutive tasks, the models fine-tuned with the HFT method consistently outperform the FFT models in terms of overall performance. This further confirms the effectiveness of HFT in preserving the original capabilities of the model and mitigating catastrophic forgetting. Furthermore, although LoRA preserves more layer parameters unchanged, it still performs worse compared to HFT. We believe this may be attributed to the low-rank decomposition resulting in a lim-
<table>
  <tr>
    <th rowspan="2">MMLU</th>
    <th colspan="3">GSM 8K</th>
    <th colspan="2">BBH</th>
    <th colspan="2">TyDi QA</th>
    <th colspan="2">Truthful QA</th>
    <th colspan="2">Human Eval</th>
    <th colspan="2">Natural Question</th>
    <th colspan="2">Trivia QA</th>
    <th colspan="2">Hotpot QA</th>
    <th colspan="2">Overall</th>
  </tr>
  <tr>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
    <th>MM</th>
    <th>LM</th>
    <th>QA</th>
  </tr>
  <tr>
    <td>SeqFT-7b</td>
    <td>35.5</td>
    <td>3.0</td>
    <td>24.3</td>
    <td>39.1</td>
    <td>42.7</td>
    <td>0.3</td>
    <td>10.0</td>
    <td>23.9</td>
    <td>14.0</td>
    <td>21.4</td>
  </tr>
  <tr>
    <td>GEM-7b</td>
    <td>40.1</td>
    <td>3.5</td>
    <td>17.0</td>
    <td>33.4</td>
    <td>41.4</td>
    <td>2.2</td>
    <td>10.0</td>
    <td>19.6</td>
    <td>14.0</td>
    <td>20.1</td>
  </tr>
  <tr>
    <td>Replay-7b</td>
    <td>45.9</td>
    <td>4.5</td>
    <td>35.2</td>
    <td>41.6</td>
    <td>39.6</td>
    <td>8.5</td>
    <td>11.6</td>
    <td>36.1</td>
    <td>14.2</td>
    <td>26.4</td>
  </tr>
  <tr>
    <td>LoraSeqFT-7b</td>
    <td>43.3</td>
    <td>11.0</td>
    <td>30.7</td>
    <td>35.5</td>
    <td>41.7</td>
    <td>8.8</td>
    <td>8.7</td>
    <td>24.7</td>
    <td>13.4</td>
    <td>24.2</td>
  </tr>
  <tr>
    <td>SeqFT-7b (Half)</td>
    <td>44.1</td>
    <td>3.5</td>
    <td>30.8</td>
    <td>41.1</td>
    <td>41.8</td>
    <td>1.6</td>
    <td>11.3</td>
    <td>38.9</td>
    <td>14.4</td>
    <td>25.3<sup>**</sup></td>
  </tr>
  <tr>
    <td>GEM-7b (Half)</td>
    <td>45.1</td>
    <td>5.0</td>
    <td>32.3</td>
    <td>34.9</td>
    <td>43.0</td>
    <td>2.7</td>
    <td>10.4</td>
    <td>35.9</td>
    <td>13.7</td>
    <td>24.8<sup>**</sup></td>
  </tr>
  <tr>
    <td>Replay-7b (Half)</td>
    <td>47.9</td>
    <td>11.0</td>
    <td>38.8</td>
    <td>42.6</td>
    <td>42.5</td>
    <td>12.7</td>
    <td>10.7</td>
    <td>38.4</td>
    <td>12.9</td>
    <td>28.6<sup>**</sup></td>
  </tr>
  <tr>
    <td>SeqFT-13b</td>
    <td>39.7</td>
    <td>5.0</td>
    <td>27.9</td>
    <td>41.0</td>
    <td>41.4</td>
    <td>0.0</td>
    <td>12.7</td>
    <td>44.3</td>
    <td>16.3</td>
    <td>25.4</td>
  </tr>
  <tr>
    <td>Replay-13b</td>
    <td>49.0</td>
    <td>3.5</td>
    <td>40.1</td>
    <td>37.7</td>
    <td>43.1</td>
    <td>12.0</td>
    <td>12.5</td>
    <td>6.7</td>
    <td>13.3</td>
    <td>24.2</td>
  </tr>
  <tr>
    <td>GEM-13b</td>
    <td>47.2</td>
    <td>4.0</td>
    <td>37.6</td>
    <td>36.3</td>
    <td>43.0</td>
    <td>10.0</td>
    <td>10.8</td>
    <td>10.2</td>
    <td>12.1</td>
    <td>23.5</td>
  </tr>
  <tr>
    <td>LoraSeqFT-13b</td>
    <td>43.3</td>
    <td>15.0</td>
    <td>42.4</td>
    <td>43.1</td>
    <td>40.5</td>
    <td>18.2</td>
    <td>10.6</td>
    <td>37.6</td>
    <td>16.2</td>
    <td>29.7</td>
  </tr>
  <tr>
    <td>SeqFT-13b (Half)</td>
    <td>50.0</td>
    <td>7.0</td>
    <td>46.3</td>
    <td>47.2</td>
    <td>41.4</td>
    <td>11.2</td>
    <td>14.7</td>
    <td>50.6</td>
    <td>18.7</td>
    <td>31.9<sup>**</sup></td>
  </tr>
  <tr>
    <td>GEM-13b (Half)</td>
    <td>49.9</td>
    <td>9.5</td>
    <td>46.5</td>
    <td>38.2</td>
    <td>45.1</td>
    <td>18.9</td>
    <td>9.8</td>
    <td>39.7</td>
    <td>14.2</td>
    <td>30.2<sup>**</sup></td>
  </tr>
  <tr>
    <td>Replay-13b (Half)</td>
    <td>50.0</td>
    <td>10.5</td>
    <td>47.1</td>
    <td>39.6</td>
    <td>45.8</td>
    <td>20.1</td>
    <td>10.1</td>
    <td>41.1</td>
    <td>14.0</td>
    <td>30.9<sup>**</sup></td>
  </tr>
</table>

Table 12: General abilities and basic knowledge performance of the final round models fine-tuned on TRACE. We compare four different fine-tuning methods and our HFT approach start from LLAMA 2-CHAT-7B and LLAMA 2-CHAT-13B.

<table>
  <tr>
    <th>FFT (linear,1e-5)</th>
    <th>FFT (linear,2e-5)</th>
    <th>HFT (linear,2e-5)</th>
    <th>FFT (cosine,2e-5)</th>
    <th>HFT (cosine,2e-5)</th>
  </tr>
  <tr>
    <td>MMLU (EM, 0-shot)</td>
    <td>49.2</td>
    <td>48.5</td>
    <td><b>50.8</b></td>
    <td>47.8</td>
    <td>50.6</td>
  </tr>
  <tr>
    <td>GSM (ACC, 8-shot)</td>
    <td>24.5</td>
    <td>25.0</td>
    <td>30.5</td>
    <td>25.5</td>
    <td>31.5</td>
  </tr>
  <tr>
    <td>BBH (EM, 0-shot)</td>
    <td>41.8</td>
    <td>42.2</td>
    <td><b>43.6</b></td>
    <td>42.2</td>
    <td><b>44.4</b></td>
  </tr>
  <tr>
    <td>TyDiQA (F1, 1-shot)</td>
    <td>51.5</td>
    <td>51.2</td>
    <td><b>52.3</b></td>
    <td>51.2</td>
    <td><b>52.8</b></td>
  </tr>
  <tr>
    <td>TruthfulQA (MC2, 0-shot)</td>
    <td>40.2</td>
    <td>41.7</td>
    <td><b>45.4</b></td>
    <td>42.6</td>
    <td><b>46.4</b></td>
  </tr>
  <tr>
    <td>HumanEval (Pass@10)</td>
    <td>36.0</td>
    <td><b>36.9</b></td>
    <td>34.6</td>
    <td>34.3</td>
    <td>33.7</td>
  </tr>
  <tr>
    <th>Overall (General Ability)</th>
    <th>40.4</th>
    <th>41.0</th>
    <th><b>42.9</b></th>
    <th>40.6</th>
    <th><b>43.2</b></th>
  </tr>
  <tr>
    <td>NaturalQuestion (EM, 0-shot)</td>
    <td>4.9</td>
    <td>3.2</td>
    <td>6.2</td>
    <td>3.5</td>
    <td>6.4</td>
  </tr>
  <tr>
    <td>TriviaQA (EM, 0-shot)</td>
    <td>22.7</td>
    <td>26.4</td>
    <td><b>32.8</b></td>
    <td>27.6</td>
    <td><b>33.6</b></td>
  </tr>
  <tr>
    <td>HotpotQA (EM, 0-shot)</td>
    <td>13.4</td>
    <td>14.5</td>
    <td><b>15.4</b></td>
    <td>13.1</td>
    <td>14.7</td>
  </tr>
  <tr>
    <th>Overall (World Knowledge)</th>
    <th>13.7</th>
    <th>14.7</th>
    <th><b>18.1</b></th>
    <th>14.7</th>
    <th><b>18.2</b></th>
  </tr>
  <tr>
    <th>Overall</th>
    <th>31.5</th>
    <th>32.2</th>
    <th><b>34.6</b></th>
    <th>32.0</th>
    <th><b>34.9</b></th>
  </tr>
</table>

Table 13: General abilities and basic knowledge of LLAMA 2 7b based on different learning rates.

ited number of trainable parameters. Merging the LoRA weights back into the original model could potentially disrupt the original parameter space to a greater extent.

A.4.6 The Impact of Learning Rates

To validate whether our approach indeed leverages the frozen parameters to mitigate the catastrophic forgetting, rather than being equivalent to the effects brought about by a reduced learning rate, we compare the half learning rate and the cosine learning rate schedule to demonstrate further that the way HFT alleviates forgetting is not depending on learning rate but is indeed due to the role played by the frozen parameters. As shown in Table 13, we observe that upon halving the learning rate, the overall performance declines, with no significant recovery in the performance on world knowledge, thereby underscoring the capability of HFT in mitigating catastrophic forgetting. Moreover, under the cosine learning rate schedule, HFT still outperforms FFT, which also demonstrates the robustness of HFT to variations in the learning rate.

A.4.7 The Impact of Randomness

Here, we discuss a series of factors related to the randomness of HFT, including different trainable parameter ratios and selection methods. Note that in the continual learning setting, we randomly select trainable parameters for each fine-tuning process, with a total of 8 random selections. The significant performance improvement of HFT over FFT indicates that it is not sensitive to fine-grained parameter selection. For all that, we also supplement a randomness experiment under the instruction tuning setting with 5 different random seeds (i.e. parameter selections). As shown in Table 14, among these 5 trials, HFT exhibits minimal variations and a stable lead relative to FFT, demonstrating its robustness again.
<table>
  <tr>
    <th></th>
    <th>HFT (seed 1)</th>
    <th>HFT (seed 2)</th>
    <th>HFT (seed 3)</th>
    <th>HFT (seed 4)</th>
    <th>HFT (seed 5)</th>
  </tr>
  <tr>
    <td>MMLU (EM, 0-shot)</td>
    <td>50.8</td>
    <td>49.9</td>
    <td>50.2</td>
    <td><b>51.2</b></td>
    <td>50.5</td>
  </tr>
  <tr>
    <td>GSM (ACC, 8-shot)</td>
    <td>30.5</td>
    <td><b>31.0</b></td>
    <td>30.5</td>
    <td>28.5</td>
    <td>29.5</td>
  </tr>
  <tr>
    <td>BBH (EM, 0-shot)</td>
    <td>43.6</td>
    <td>43.2</td>
    <td>42.9</td>
    <td>43.4</td>
    <td><b>44.1</b></td>
  </tr>
  <tr>
    <td>TyDiQA (F1, 1-shot)</td>
    <td>52.3</td>
    <td>52.3</td>
    <td><b>53.2</b></td>
    <td><b>52.8</b></td>
    <td>51.7</td>
  </tr>
  <tr>
    <td>TruthfulQA (MC2, 0-shot)</td>
    <td><b>45.4</b></td>
    <td><b>45.7</b></td>
    <td>44.7</td>
    <td>45.2</td>
    <td>44.9</td>
  </tr>
  <tr>
    <td>HumanEval (Pass@10)</td>
    <td>34.6</td>
    <td><b>35.1</b></td>
    <td>34.8</td>
    <td>34.7</td>
    <td><b>35.2</b></td>
  </tr>
  <tr>
    <th colspan="6"><i>Overall (General Ability)</i></th>
    <th>42.9</th>
    <th>42.9</th>
    <th>42.7</th>
    <th>42.6</th>
    <th>42.7</th>
  </tr>
  <tr>
    <td>NaturalQuestion (EM, 0-shot)</td>
    <td>6.2</td>
    <td>6.1</td>
    <td>5.9</td>
    <td>6.1</td>
    <td><b>6.4</b></td>
  </tr>
  <tr>
    <td>TriviaQA (EM, 0-shot)</td>
    <td>32.8</td>
    <td>31.9</td>
    <td><b>33.4</b></td>
    <td><b>33.1</b></td>
    <td>33.0</td>
  </tr>
  <tr>
    <td>HotpotQA (EM, 0-shot)</td>
    <td>15.4</td>
    <td>15.4</td>
    <td><b>15.6</b></td>
    <td>14.9</td>
    <td><b>15.6</b></td>
  </tr>
  <tr>
    <th colspan="6"><i>Overall (World Knowledge)</i></th>
    <th>18.1</th>
    <th>17.8</th>
    <th>18.3</th>
    <th>18.0</th>
    <th>18.3</th>
  </tr>
  <tr>
    <th colspan="6"><i>Overall</i></th>
    <th><b>34.6</b></th>
    <th><b>34.5</b></th>
    <th><b>34.6</b></th>
    <th><b>34.4</b></th>
    <th><b>34.6</b></th>
  </tr>
</table>

Table 14: General abilities and basic knowledge of LLAMA 2 7B based on different random seeds.

<table>
  <tr>
    <th>Models</th>
    <th>AlpacaEval 2.0</th>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT</td>
    <td><b>6.96</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT (Reset)</td>
    <td>2.98</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-SFT (Half)</td>
    <td>5.59</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO</td>
    <td><b>10.68</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO (Reset)</td>
    <td>8.44</td>
  </tr>
  <tr>
    <td>LLAMA 2-7B-DPO (Half)</td>
    <td>9.07</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT</td>
    <td>8.32</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT (Reset)</td>
    <td><b>11.93</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-SFT (Half)</td>
    <td>10.43</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO</td>
    <td>11.55</td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO (Reset)</td>
    <td><b>12.55</b></td>
  </tr>
  <tr>
    <td>LLAMA 2-13B-DPO (Half)</td>
    <td>11.68</td>
  </tr>
</table>

Table 15: Evaluation results on AlpacaEval 2.0.

<table>
  <tr>
    <th>Method</th>
    <th>Dataset</th>
    <th colspan="4">IFEval</th>
  </tr>
  <tr>
    <td></td>
    <td></td>
    <td>Pr.S</td>
    <td>In.S</td>
    <td>Pr.L</td>
    <td>In.L</td>
  </tr>
  <tr>
    <th colspan="6">Results of Llama-3.1-8B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-IF</td>
    <td>61.00</td>
    <td>71.58</td>
    <td>65.25</td>
    <td>75.42</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-IF</td>
    <td><b>67.10</b></td>
    <td><b>75.30</b></td>
    <td><b>70.43</b></td>
    <td><b>78.30</b></td>
  </tr>
  <tr>
    <th colspan="6">Results of Qwen-2.5-7B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-IF</td>
    <td>34.57</td>
    <td>49.64</td>
    <td>38.45</td>
    <td>52.52</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-IF</td>
    <td><b>37.89</b></td>
    <td><b>53.00</b></td>
    <td><b>39.37</b></td>
    <td><b>54.56</b></td>
  </tr>
</table>

Table 16: More results on recent models and datasets with Tulu3-IF.

<table>
  <tr>
    <th>Method</th>
    <th>Dataset</th>
    <th>HumanEval</th>
    <th>MBPP</th>
    <th>LiveCode Bench</th>
  </tr>
  <tr>
    <th colspan="5">Results of Llama-3.1-8B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-Code</td>
    <td>43.90</td>
    <td>42.60</td>
    <td>8.35</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-Code</td>
    <td><b>51.83</b></td>
    <td><b>47.80</b></td>
    <td><b>13.99</b></td>
  </tr>
  <tr>
    <th colspan="5">Results of Qwen-2.5-7B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-Code</td>
    <td>72.56</td>
    <td>63.80</td>
    <td>21.92</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-Code</td>
    <td><b>74.39</b></td>
    <td><b>66.80</b></td>
    <td><b>24.43</b></td>
  </tr>
</table>

Table 17: More results on recent models and datasets with Tulu3-Code.

<table>
  <tr>
    <th>Method</th>
    <th>Dataset</th>
    <th>GSM8K</th>
    <th>MATH</th>
  </tr>
  <tr>
    <th colspan="4">Results of Llama-3.1-8B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-Math</td>
    <td>72.63</td>
    <td>25.32</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-Math</td>
    <td><b>74.37</b></td>
    <td><b>26.58</b></td>
  </tr>
  <tr>
    <th colspan="4">Results of Qwen-2.5-7B</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>Tulu3-Math</td>
    <td>84.91</td>
    <td>46.34</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>Tulu3-Math</td>
    <td><b>85.82</b></td>
    <td><b>50.24</b></td>
  </tr>
</table>

Table 18: More results on recent models and datasets with Tulu3-Math.

A.4.8 Evaluation on AlpacaEval

As shown in Table 15, we evaluate different models on AlpacaEval 2.0. The results indicate that our method is less effective than FFT on LLAMA 2-7B. However, a reversal occurs when the model size scales up to 13b, where our approach outperforms the FFT models comprehensively. This suggests that our method has greater potential on much larger-scale LLMs, as supported by the experimental results in Table 1, which show a larger improvement of HFT compared to FFT on LLAMA 2-13B compared to LLAMA 2-7B. Interestingly, the Half-Reset method performs well on LLAMA 2-13B but shows completely different results on LLAMA 2-7B. This suggests that simply resetting half of the parameters may not provide consistent performance since the model is trained on the full set of parameters.
<table>
  <tr>
    <th>Method</th>
    <th>Dataset</th>
    <th>MMLU</th>
    <th>Hellaswag</th>
    <th>GSM8K</th>
    <th>HumanEval</th>
    <th>BBH</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td colspan="10"><i>Results of Llama-3.1-8B</i></td>
  </tr>
  <tr>
    <td>FFT</td>
    <td>WizardLM-143K</td>
    <td><b>58.69</b></td>
    <td><b>65.10</b></td>
    <td>57.62</td>
    <td>48.17</td>
    <td>57.71</td>
    <td>26.90</td>
    <td>52.69</td>
    <td>52.41</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>WizardLM-143K</td>
    <td>58.23</td>
    <td>65.07</td>
    <td><b>62.77</b></td>
    <td><b>50.61</b></td>
    <td><b>59.19</b></td>
    <td><b>29.72</b></td>
    <td><b>59.42</b></td>
    <td><b>55.00</b></td>
  </tr>
  <tr>
    <td colspan="10"><i>Results of Qwen-2.5-7B</i></td>
  </tr>
  <tr>
    <td>FFT</td>
    <td>WizardLM-143K</td>
    <td>57.09</td>
    <td><b>77.41</b></td>
    <td>81.35</td>
    <td><b>77.82</b></td>
    <td>67.46</td>
    <td>24.18</td>
    <td>54.95</td>
    <td>62.89</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>WizardLM-143K</td>
    <td><b>63.35</b></td>
    <td>76.77</td>
    <td><b>81.96</b></td>
    <td>75.67</td>
    <td><b>67.73</b></td>
    <td><b>25.01</b></td>
    <td><b>57.30</b></td>
    <td><b>63.97</b></td>
  </tr>
</table>

Table 19: More results on recent models and datasets with WizardLM-143K.

A.4.9 More Results on Recent Models and Datasets

To further validate the effectiveness of HFT, we evaluate the performance of recent advanced models (Qwen-2.5-7B (Qwen et al., 2025) and Llama-3.1-8B) on four subsets of Tulu3 (Lambert et al., 2025) (Tulu3-instruction-following, Tulu3-math, Tulu3-code), and WizardLM-143K (Xu et al., 2023a). According to the technical reports of the Llama-3.1 and Qwen-2.5 series models, we train them for two epochs using learning rates of 1e-5 and 7e-6, respectively. The evaluation is conducted using the OpenCompass (Contributors, 2023) framework, and the results are shown in Tables 16, 17, 18, 19. We introduce three benchmarks including MATH (Hendrycks et al., 2021b), LiveCodeBench (Jain et al., 2024), MBPP (Austin et al., 2021) and IFEval (Zhou et al., 2023b) for more convincing evaluations. The experimental results on the state-of-the-art models trained with four different domains demonstrate that HFT not only mitigates forgetting but also achieves superior performance in broad downstream tasks compared to FFT. We believe these additional experiments further demonstrate the superiority of the HFT method, and we will also enhance the related experiments in the final version of the paper to improve its completeness and timeliness.

A.4.10 The Training Stability of HFT

To further verify the training stability of HFT and address concerns that FFT might underperform compared to HFT due to overfitting, we train Llama-3.1-8B and Qwen-2.5-7B on the WizardLM-143K dataset and evaluate models at different training steps. As shown in Table 20 and 21, we find that FFT does not exhibit overfitting. During continual fine-tuning, the average performance of both HFT and FFT improves; however, at each step, the average performance of HFT is better than that of FFT on Llama-3.1-8B. On Qwen-2.5-7B, FFT exhibits instability during training, with significant fluctuations in performance across different steps, whereas HFT ensures smooth and stable training. This phenomenon further demonstrates the superiority of HFT, which enables more stable training and alleviates forgetting.

A.4.11 The Exploration of Dynamic Freezing Strategy

In this section, we explore a new strategy that dynamically freezes the parameters during fine-tuning. Specifically, at the start of training, we train all parameters, and as the training progresses, we continuously freeze parameters in the model (freezing 10% of the parameters every time training progresses by 10%). We train Llama-3.1-8B on the WizardLM-143K dataset, and the results are shown in Table 22. We find that dynamically freezing parameters performs better than FFT but worse than HFT. We believe that maintaining the overall structural consistency of the model during training is important, and dynamically freezing parameters can disrupt such consistency.

A.4.12 Explore the Importance of Different Layers

In this section, we mainly focus on the importance of different layers in self-attention blocks. Specifically, we train Llama-3.1-8B on WizardLM-143K. Since the Llama-3 series models use GQA technology, we group \( Q \) and \( O \), and \( K \) and \( V \), selecting them separately. Currently, we have explored two sets of experiments: one where \( Q \) and \( K \) are fixed as frozen, and another where \( O \) and \( V \) are fixed as frozen. The specific results are shown in Table 23. We find that if only the \( Q \) and \( K \) are frozen, or only the \( O \) and \( V \) are frozen, although state-of-the-art performance is achieved on some benchmarks, severe degradation occurs in certain evaluations, leading to subpar average performance. This further highlights that the fine-grained parameter selection approach adopted in HFT can maintain the
<table>
  <tr>
    <th>Steps</th>
    <th>MMLU</th>
    <th>Hellaswag</th>
    <th>GSM8K</th>
    <th>HumanEval</th>
    <th>BBH</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td colspan="9"><i>Results of FFT</i></td>
  </tr>
  <tr>
    <td>448</td>
    <td>47.72</td>
    <td>33.07</td>
    <td>50.27</td>
    <td>37.90</td>
    <td>52.85</td>
    <td>27.01</td>
    <td>50.65</td>
    <td>42.77</td>
  </tr>
  <tr>
    <td>896</td>
    <td>45.83</td>
    <td>51.60</td>
    <td>50.42</td>
    <td>42.07</td>
    <td>54.16</td>
    <td>27.31</td>
    <td>53.37</td>
    <td>46.39</td>
  </tr>
  <tr>
    <td>1344</td>
    <td><b>51.04</b></td>
    <td>53.37</td>
    <td>55.72</td>
    <td>47.84</td>
    <td>58.03</td>
    <td>25.37</td>
    <td>52.45</td>
    <td>49.12</td>
  </tr>
  <tr>
    <td>1792</td>
    <td><b>57.76</b></td>
    <td><b>65.41</b></td>
    <td>56.41</td>
    <td>50.61</td>
    <td><b>58.12</b></td>
    <td>27.37</td>
    <td>53.58</td>
    <td>52.75</td>
  </tr>
  <tr>
    <td>2234</td>
    <td><b>58.69</b></td>
    <td><b>65.10</b></td>
    <td>57.62</td>
    <td>48.17</td>
    <td>57.71</td>
    <td>26.90</td>
    <td>52.69</td>
    <td>52.41</td>
  </tr>
  <tr>
    <td colspan="9"><i>Results of HFT</i></td>
  </tr>
  <tr>
    <td>448</td>
    <td><b>50.83</b></td>
    <td><b>40.98</b></td>
    <td><b>55.88</b></td>
    <td><b>42.07</b></td>
    <td><b>55.44</b></td>
    <td><b>30.44</b></td>
    <td><b>57.72</b></td>
    <td><b>47.63</b></td>
  </tr>
  <tr>
    <td>896</td>
    <td><b>50.31</b></td>
    <td><b>56.87</b></td>
    <td><b>57.09</b></td>
    <td><b>42.67</b></td>
    <td><b>57.42</b></td>
    <td><b>30.89</b></td>
    <td><b>59.01</b></td>
    <td><b>50.61</b></td>
  </tr>
  <tr>
    <td>1344</td>
    <td>50.25</td>
    <td><b>61.77</b></td>
    <td><b>59.21</b></td>
    <td>49.62</td>
    <td><b>60.05</b></td>
    <td><b>29.78</b></td>
    <td><b>57.52</b></td>
    <td><b>52.60</b></td>
  </tr>
  <tr>
    <td>1792</td>
    <td>56.65</td>
    <td>62.89</td>
    <td><b>60.58</b></td>
    <td><b>53.05</b></td>
    <td>57.85</td>
    <td><b>30.17</b></td>
    <td><b>59.18</b></td>
    <td><b>54.34</b></td>
  </tr>
  <tr>
    <td>2234</td>
    <td>58.23</td>
    <td>65.07</td>
    <td><b>62.77</b></td>
    <td><b>50.61</b></td>
    <td><b>59.19</b></td>
    <td><b>29.72</b></td>
    <td><b>59.42</b></td>
    <td><b>55.00</b></td>
  </tr>
</table>

Table 20: The results of different training steps on Llama-3.1-8B.

<table>
  <tr>
    <th>Steps</th>
    <th>MMLU</th>
    <th>Hellaswag</th>
    <th>GSM8K</th>
    <th>HumanEval</th>
    <th>BBH</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td colspan="9"><i>Results of FFT</i></td>
  </tr>
  <tr>
    <td>448</td>
    <td>62.97</td>
    <td>50.84</td>
    <td>79.68</td>
    <td>59.45</td>
    <td>66.23</td>
    <td><b>27.17</b></td>
    <td>54.52</td>
    <td>57.27</td>
  </tr>
  <tr>
    <td>896</td>
    <td>59.50</td>
    <td><b>71.05</b></td>
    <td><b>80.67</b></td>
    <td>63.41</td>
    <td>66.35</td>
    <td>26.37</td>
    <td>52.71</td>
    <td>60.01</td>
  </tr>
  <tr>
    <td>1344</td>
    <td>48.62</td>
    <td>77.38</td>
    <td>81.35</td>
    <td>61.59</td>
    <td><b>68.00</b></td>
    <td>23.80</td>
    <td>54.33</td>
    <td>59.30</td>
  </tr>
  <tr>
    <td>1792</td>
    <td>54.66</td>
    <td>79.22</td>
    <td><b>81.80</b></td>
    <td>71.89</td>
    <td>67.38</td>
    <td>24.46</td>
    <td>54.32</td>
    <td>61.96</td>
  </tr>
  <tr>
    <td>2234</td>
    <td>57.09</td>
    <td><b>77.41</b></td>
    <td>81.35</td>
    <td><b>77.82</b></td>
    <td>67.46</td>
    <td>24.18</td>
    <td>54.95</td>
    <td>62.89</td>
  </tr>
  <tr>
    <td colspan="9"><i>Results of HFT</i></td>
  </tr>
  <tr>
    <td>448</td>
    <td><b>64.67</b></td>
    <td><b>51.02</b></td>
    <td><b>80.44</b></td>
    <td><b>60.68</b></td>
    <td><b>67.76</b></td>
    <td>27.09</td>
    <td><b>56.70</b></td>
    <td><b>58.34</b></td>
  </tr>
  <tr>
    <td>896</td>
    <td><b>64.23</b></td>
    <td>67.35</td>
    <td><b>80.67</b></td>
    <td><b>71.34</b></td>
    <td><b>67.37</b></td>
    <td><b>27.01</b></td>
    <td><b>56.97</b></td>
    <td><b>62.13</b></td>
  </tr>
  <tr>
    <td>1344</td>
    <td><b>63.79</b></td>
    <td><b>79.39</b></td>
    <td><b>81.43</b></td>
    <td><b>64.02</b></td>
    <td>67.95</td>
    <td><b>25.10</b></td>
    <td><b>57.68</b></td>
    <td><b>62.77</b></td>
  </tr>
  <tr>
    <td>1792</td>
    <td><b>63.84</b></td>
    <td><b>81.52</b></td>
    <td>81.35</td>
    <td><b>72.18</b></td>
    <td><b>67.62</b></td>
    <td><b>24.85</b></td>
    <td><b>57.55</b></td>
    <td><b>64.13</b></td>
  </tr>
  <tr>
    <td>2234</td>
    <td><b>63.35</b></td>
    <td>76.77</td>
    <td><b>81.96</b></td>
    <td>75.67</td>
    <td><b>67.73</b></td>
    <td><b>25.01</b></td>
    <td><b>57.30</b></td>
    <td><b>63.97</b></td>
  </tr>
</table>

Table 21: The results of different training steps on Qwen-2.5-7B.

<table>
  <tr>
    <th>Model</th>
    <th>MMLU</th>
    <th>Hellaswag</th>
    <th>GSM8K</th>
    <th>HumanEval</th>
    <th>BBH</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>58.69</td>
    <td><b>65.10</b></td>
    <td>57.62</td>
    <td>48.17</td>
    <td>57.71</td>
    <td>26.90</td>
    <td>52.69</td>
    <td>52.41</td>
  </tr>
  <tr>
    <td>Dynamic</td>
    <td><b>59.13</b></td>
    <td>64.32</td>
    <td><b>63.38</b></td>
    <td>47.82</td>
    <td>58.15</td>
    <td>28.01</td>
    <td>57.62</td>
    <td>54.06</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>58.23</td>
    <td>65.07</td>
    <td>62.77</td>
    <td><b>50.61</b></td>
    <td><b>59.19</b></td>
    <td><b>29.72</b></td>
    <td><b>59.42</b></td>
    <td><b>55.00</b></td>
  </tr>
</table>

Table 22: The results of the dynamically freezing strategy on Llama-3.1-8B.

<table>
  <tr>
    <th>Model</th>
    <th>MMLU</th>
    <th>Hellaswag</th>
    <th>GSM8K</th>
    <th>HumanEval</th>
    <th>BBH</th>
    <th>NQ</th>
    <th>TriviaQA</th>
    <th>Avg.</th>
  </tr>
  <tr>
    <td>FFT</td>
    <td>58.69</td>
    <td>65.10</td>
    <td>57.62</td>
    <td>48.17</td>
    <td>57.71</td>
    <td>26.90</td>
    <td>52.69</td>
    <td>52.41</td>
  </tr>
  <tr>
    <td>Freeze Q/K</td>
    <td>40.12</td>
    <td><b>65.13</b></td>
    <td>60.50</td>
    <td>49.39</td>
    <td>55.35</td>
    <td>28.45</td>
    <td>57.92</td>
    <td>50.98</td>
  </tr>
  <tr>
    <td>Freeze O/V</td>
    <td>45.16</td>
    <td>64.41</td>
    <td><b>63.15</b></td>
    <td>38.41</td>
    <td>48.84</td>
    <td><b>30.06</b></td>
    <td><b>60.86</b></td>
    <td>50.13</td>
  </tr>
  <tr>
    <td>HFT</td>
    <td>58.23</td>
    <td>65.07</td>
    <td>62.77</td>
    <td><b>50.61</b></td>
    <td><b>59.19</b></td>
    <td><b>29.72</b></td>
    <td><b>59.42</b></td>
    <td><b>55.00</b></td>
  </tr>
</table>

Table 23: The results of freezing different layers on Llama-3.1-8B.
overall structural consistency of the model, thereby achieving more balanced performance.

A.4.13  Detailed Results of Revisiting Embedding and LM_Head Layers
Table 24 details the results of freezing the input and output layers. Meanwhile, Table 25 and 26 show the detailed results of the two adjacent numbers of parameter settings on TRACE.

A.4.14  Detailed Results of Different Parameter Selection Strategies
Table 27 and 28 provide the detailed results on TRACE with model-level and layer-level parameter selection strategies mentioned in Section 4.3.

A.4.15  Detailed Results of TRACE
Table 29 to 42 show the detailed results of different models and approaches of each round during the continual learning on TRACE.
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>50.1</td>
    <td>48.0</td>
    <td>47.2</td>
    <td>45.8</td>
    <td>46.4</td>
    <td>46.2</td>
    <td>46.3</td>
    <td>48.0</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>69.0</td>
    <td>66.1</td>
    <td>65.7</td>
    <td>65.7</td>
    <td>64.7</td>
    <td>63.9</td>
    <td>66.9</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>37.5</td>
    <td>34.5</td>
    <td>34.2</td>
    <td>32.7</td>
    <td>31.9</td>
    <td>33.2</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>51.2</td>
    <td>50.3</td>
    <td>49.8</td>
    <td>49.2</td>
    <td>50.8</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>58.1</td>
    <td>58.0</td>
    <td>56.8</td>
    <td>56.2</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>33.3</td>
    <td>25.9</td>
    <td>29.6</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>45.8</td>
    <td>43.1</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.6</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>50.1</b></th>
    <th><b>58.5</b></th>
    <th><b>50.3</b></th>
    <th><b>49.3</b></th>
    <th><b>50.9</b></th>
    <th><b>47.5</b></th>
    <th><b>45.7</b></th>
    <th><b>46.1</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-2.2%</b></th>
  </tr>
</table>

Table 24: Detailed results on TRACE with 50.0% trainable parameters while freezing embedding and lm_head layers.

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>49.2</td>
    <td>43.7</td>
    <td>43.2</td>
    <td>44.2</td>
    <td>44.2</td>
    <td>44.4</td>
    <td>43.7</td>
    <td>45.1</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>71.0</td>
    <td>64.3</td>
    <td>65.3</td>
    <td>60.7</td>
    <td>65.9</td>
    <td>65.1</td>
    <td>63.3</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>46.9</td>
    <td>37.7</td>
    <td>35.4</td>
    <td>39.0</td>
    <td>38.5</td>
    <td>36.9</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.9</td>
    <td>52.6</td>
    <td>53.6</td>
    <td>53.6</td>
    <td>53.4</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>85.7</td>
    <td>77.5</td>
    <td>71.8</td>
    <td>74.8</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>33.3</td>
    <td>29.6</td>
    <td>33.3</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>56.6</td>
    <td>48.9</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.1</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>49.2</b></th>
    <th><b>57.4</b></th>
    <th><b>51.5</b></th>
    <th><b>51.3</b></th>
    <th><b>55.7</b></th>
    <th><b>52.3</b></th>
    <th><b>51.3</b></th>
    <th><b>49.6</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-5.6%</b></th>
  </tr>
</table>

Table 25: Detailed results on TRACE with 38.9% trainable parameters while updating embedding and lm_head layers.

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>45.3</td>
    <td>50.8</td>
    <td>50.9</td>
    <td>51.4</td>
    <td>51.3</td>
    <td>51.4</td>
    <td>51.1</td>
    <td>53.3</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>72.8</td>
    <td>63.7</td>
    <td>65.7</td>
    <td>6.3</td>
    <td>68.3</td>
    <td>69.0</td>
    <td>67.9</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>48.9</td>
    <td>41.1</td>
    <td>38.3</td>
    <td>41.3</td>
    <td>41.1</td>
    <td>40.0</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.3</td>
    <td>50.3</td>
    <td>52.8</td>
    <td>52.9</td>
    <td>52.9</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>88.2</td>
    <td>70.6</td>
    <td>67.3</td>
    <td>69.4</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>30.9</td>
    <td>28.4</td>
    <td>21.0</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>59.4</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.8</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>45.3</b></th>
    <th><b>61.8</b></th>
    <th><b>54.5</b></th>
    <th><b>53.9</b></th>
    <th><b>46.9</b></th>
    <th><b>52.6</b></th>
    <th><b>52.7</b></th>
    <th><b>49.9</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-5.6%</b></th>
  </tr>
</table>

Table 26: Detailed results on TRACE with 61.1% trainable parameters while updating embedding and lm_head layers.
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>49.3</td>
    <td>49.1</td>
    <td>48.8</td>
    <td>50.2</td>
    <td>50.0</td>
    <td>48.9</td>
    <td>48.1</td>
    <td>49.2</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>70.6</td>
    <td>57.5</td>
    <td>53.8</td>
    <td>42.7</td>
    <td>54.4</td>
    <td>58.1</td>
    <td>55.2</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>48.9</td>
    <td>37.8</td>
    <td>36.5</td>
    <td>38.2</td>
    <td>37.3</td>
    <td>38.9</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.7</td>
    <td>55.4</td>
    <td>55.9</td>
    <td>54.8</td>
    <td>55.7</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>87.7</td>
    <td>59.8</td>
    <td>54.2</td>
    <td>56.4</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>38.3</td>
    <td>22.2</td>
    <td>25.9</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>55.7</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.7</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>49.3</b></th>
    <th><b>59.9</b></th>
    <th><b>51.7</b></th>
    <th><b>49.9</b></th>
    <th><b>54.5</b></th>
    <th><b>49.3</b></th>
    <th><b>47.2</b></th>
    <th><b>46.9</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-9.2%</b></th>
  </tr>
</table>

Table 27: Detailed results on TRACE with model-level parameter selection.

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>50.8</td>
    <td>41.4</td>
    <td>44.6</td>
    <td>46.5</td>
    <td>47.5</td>
    <td>48.6</td>
    <td>48.2</td>
    <td>49.0</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>72.2</td>
    <td>58.5</td>
    <td>54.6</td>
    <td>1.8</td>
    <td>46.8</td>
    <td>50.2</td>
    <td>50.0</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>47.1</td>
    <td>34.7</td>
    <td>34.5</td>
    <td>37.2</td>
    <td>38.6</td>
    <td>37.1</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>56.5</td>
    <td>53.3</td>
    <td>53.8</td>
    <td>54.2</td>
    <td>54.1</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>88.5</td>
    <td>84.4</td>
    <td>76.2</td>
    <td>77.5</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>35.8</td>
    <td>28.4</td>
    <td>21.0</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.2</td>
    <td>52.9</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.5</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>50.8</b></th>
    <th><b>56.8</b></th>
    <th><b>50.1</b></th>
    <th><b>48.1</b></th>
    <th><b>45.1</b></th>
    <th><b>51.1</b></th>
    <th><b>50.4</b></th>
    <th><b>47.9</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-8.3%</b></th>
  </tr>
</table>

Table 28: Detailed results on TRACE with layer-level parameter selection.

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>48.5</td>
    <td>49.7</td>
    <td>48.5</td>
    <td>48.3</td>
    <td>6.7</td>
    <td>47.4</td>
    <td>47.2</td>
    <td>48.7</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>71.6</td>
    <td>46.6</td>
    <td>46.4</td>
    <td>0.4</td>
    <td>43.1</td>
    <td>42.9</td>
    <td>44.0</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>49.0</td>
    <td>39.9</td>
    <td>40.8</td>
    <td>37.6</td>
    <td>34.5</td>
    <td>37.9</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.0</td>
    <td>49.2</td>
    <td>54.5</td>
    <td>54.2</td>
    <td>54.0</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>89.1</td>
    <td>71.5</td>
    <td>44.6</td>
    <td>60.6</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>30.9</td>
    <td>24.7</td>
    <td>25.9</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>59.4</td>
    <td>52.6</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.5</td>
  </tr>
  <tr>
    <th>OP</th>
    <th><b>48.5</b></th>
    <th><b>60.7</b></th>
    <th><b>48.0</b></th>
    <th><b>47.9</b></th>
    <th><b>37.2</b></th>
    <th><b>47.5</b></th>
    <th><b>43.9</b></th>
    <th><b>45.7</b></th>
  </tr>
  <tr>
    <th>BWT</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th>-</th>
    <th><b>-10.2%</b></th>
  </tr>
</table>

Table 29: Detailed results on TRACE with SeqFT (start from LLAMA 2-CHAT-7B).
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>49.4</td>
    <td>47.6</td>
    <td>45.6</td>
    <td>46.4</td>
    <td>47.8</td>
    <td>49.5</td>
    <td>49.1</td>
    <td>49.3</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>71.8</td>
    <td>57.7</td>
    <td>59.1</td>
    <td>46.0</td>
    <td>66.5</td>
    <td>67.3</td>
    <td>66.3</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>47.4</td>
    <td>39.1</td>
    <td>31.2</td>
    <td>38.6</td>
    <td>38.4</td>
    <td>35.7</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.4</td>
    <td>52.1</td>
    <td>54.8</td>
    <td>55.0</td>
    <td>55.0</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>87.4</td>
    <td>82.1</td>
    <td>77.6</td>
    <td>75.3</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>42.0</td>
    <td>30.9</td>
    <td>32.1</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>58.5</td>
    <td>55.1</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.3</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>49.4</b></td>
    <td><b>59.7</b></td>
    <td><b>50.2</b></td>
    <td><b>50.5</b></td>
    <td><b>52.9</b></td>
    <td><b>55.6</b></td>
    <td><b>53.8</b></td>
    <td><b>51.3</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-5.6%</b></td>
  </tr>
</table>

Table 30: Detailed results on TRACE with SeqFT and HFT (start from LLAMA 2-CHAT-7B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>50.0</td>
    <td>48.9</td>
    <td>48.4</td>
    <td>47.7</td>
    <td>13.0</td>
    <td>46.5</td>
    <td>45.7</td>
    <td>48.1</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>69.4</td>
    <td>60.3</td>
    <td>59.7</td>
    <td>0.4</td>
    <td>56.5</td>
    <td>57.1</td>
    <td>58.5</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>49.0</td>
    <td>40.4</td>
    <td>38.4</td>
    <td>38.8</td>
    <td>34.8</td>
    <td>39.0</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>56.7</td>
    <td>51.2</td>
    <td>54.0</td>
    <td>53.6</td>
    <td>53.8</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>89.5</td>
    <td>64.2</td>
    <td>29.5</td>
    <td>54.5</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>33.3</td>
    <td>32.1</td>
    <td>33.3</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>59.7</td>
    <td>57.2</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.8</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>50.0</b></td>
    <td><b>59.2</b></td>
    <td><b>52.6</b></td>
    <td><b>51.1</b></td>
    <td><b>38.5</b></td>
    <td><b>48.9</b></td>
    <td><b>44.6</b></td>
    <td><b>48.2</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-7.9%</b></td>
  </tr>
</table>

Table 31: Detailed results on TRACE with GEM (start from LLAMA 2-CHAT-7B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>50.3</td>
    <td>49.0</td>
    <td>47.0</td>
    <td>48.3</td>
    <td>50.0</td>
    <td>50.7</td>
    <td>50.1</td>
    <td>51.3</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>70.0</td>
    <td>58.9</td>
    <td>60.1</td>
    <td>36.1</td>
    <td>63.9</td>
    <td>65.9</td>
    <td>65.5</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>47.5</td>
    <td>40.2</td>
    <td>38.2</td>
    <td>39.2</td>
    <td>39.0</td>
    <td>37.9</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.0</td>
    <td>53.0</td>
    <td>55.3</td>
    <td>55.1</td>
    <td>54.6</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>88.4</td>
    <td>76.8</td>
    <td>70.1</td>
    <td>68.4</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>34.6</td>
    <td>24.7</td>
    <td>29.6</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>60.0</td>
    <td>53.6</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.0</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>50.3</b></td>
    <td><b>59.5</b></td>
    <td><b>51.1</b></td>
    <td><b>51.4</b></td>
    <td><b>53.1</b></td>
    <td><b>53.4</b></td>
    <td><b>52.1</b></td>
    <td><b>50.2</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-5.9%</b></td>
  </tr>
</table>

Table 32: Detailed results on TRACE with GEM and HFT (start from LLAMA 2-CHAT-7B).
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>51.7</td>
    <td>50.1</td>
    <td>49.4</td>
    <td>48.2</td>
    <td>50.6</td>
    <td>49.7</td>
    <td>49.9</td>
    <td>52.0</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>64.9</td>
    <td>68.1</td>
    <td>70.2</td>
    <td>70.0</td>
    <td>70.0</td>
    <td>70.6</td>
    <td>70.0</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>43.4</td>
    <td>48.0</td>
    <td>46.1</td>
    <td>46.5</td>
    <td>46.4</td>
    <td>44.8</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>53.9</td>
    <td>55.0</td>
    <td>54.1</td>
    <td>54.0</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>81.9</td>
    <td>86.0</td>
    <td>86.3</td>
    <td>87.5</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>30.9</td>
    <td>32.1</td>
    <td>32.1</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>55.7</td>
    <td>53.5</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.6</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>51.7</b></td>
    <td><b>57.5</b></td>
    <td><b>53.6</b></td>
    <td><b>55.1</b></td>
    <td><b>60.7</b></td>
    <td><b>56.2</b></td>
    <td><b>56.4</b></td>
    <td><b>54.3</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><span style="color:green">1.4%</span></td>
  </tr>
</table>

Table 33: Detailed results on TRACE with Replay (start from LLAMA 2-CHAT-7B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>47.7</td>
    <td>53.5</td>
    <td>50.6</td>
    <td>51.0</td>
    <td>50.8</td>
    <td>50.2</td>
    <td>51.1</td>
    <td>52.1</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>61.1</td>
    <td>69.4</td>
    <td>70.8</td>
    <td>69.8</td>
    <td>70.2</td>
    <td>69.4</td>
    <td>69.8</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>39.3</td>
    <td>47.1</td>
    <td>47.0</td>
    <td>46.0</td>
    <td>46.7</td>
    <td>47.3</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>55.3</td>
    <td>56.3</td>
    <td>56.3</td>
    <td>56.5</td>
    <td>55.6</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>87.3</td>
    <td>52.2</td>
    <td>85.0</td>
    <td>84.8</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>37.0</td>
    <td>29.6</td>
    <td>32.1</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>48.0</td>
    <td>50.5</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.5</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>47.7</b></td>
    <td><b>57.3</b></td>
    <td><b>53.1</b></td>
    <td><b>56.1</b></td>
    <td><b>62.2</b></td>
    <td><b>52.0</b></td>
    <td><b>55.2</b></td>
    <td><b>54.1</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><span style="color:green">+2.1%</span></td>
  </tr>
</table>

Table 34: Detailed results on TRACE with Replay and HFT (start from LLAMA 2-CHAT-7B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>51.6</td>
    <td>48.1</td>
    <td>47.4</td>
    <td>46.9</td>
    <td>24.1</td>
    <td>12.0</td>
    <td>4.1</td>
    <td>7.9</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>68.8</td>
    <td>58.3</td>
    <td>52.6</td>
    <td>0.0</td>
    <td>48.4</td>
    <td>44.2</td>
    <td>1.4</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>45.7</td>
    <td>10.6</td>
    <td>5.9</td>
    <td>1.1</td>
    <td>2.7</td>
    <td>3.0</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>58.6</td>
    <td>20.8</td>
    <td>46.8</td>
    <td>45.2</td>
    <td>0.4</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>66.1</td>
    <td>50.7</td>
    <td>41.3</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>33.3</td>
    <td>27.2</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>50.5</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>38.1</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>51.6</b></td>
    <td><b>58.5</b></td>
    <td><b>50.5</b></td>
    <td><b>42.2</b></td>
    <td><b>23.4</b></td>
    <td><b>32.1</b></td>
    <td><b>30.7</b></td>
    <td><b>6.4</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><span style="color:red">-45.2%</span></td>
  </tr>
</table>

Table 35: Detailed results on TRACE with LoRASeqFT (start from LLAMA 2-CHAT-7B).
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>51.3</td>
    <td>34.9</td>
    <td>37.6</td>
    <td>40.0</td>
    <td>41.0</td>
    <td>44.2</td>
    <td>43.8</td>
    <td>44.9</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>70.0</td>
    <td>57.5</td>
    <td>52.6</td>
    <td>4.2</td>
    <td>49.0</td>
    <td>47.2</td>
    <td>49.8</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>50.5</td>
    <td>44.9</td>
    <td>44.4</td>
    <td>45.7</td>
    <td>44.7</td>
    <td>41.9</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>56.8</td>
    <td>54.9</td>
    <td>54.4</td>
    <td>53.1</td>
    <td>54.6</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>91.3</td>
    <td>73.5</td>
    <td>66.1</td>
    <td>73.9</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>43.2</td>
    <td>28.4</td>
    <td>25.9</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>62.5</td>
    <td>59.4</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.4</td>
  </tr>
  <tr>
    <th>OP</th>
    <td>51.3</td>
    <td>52.5</td>
    <td>48.5</td>
    <td>48.6</td>
    <td>47.2</td>
    <td>51.7</td>
    <td>49.4</td>
    <td>49.0</td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-9.4%</b></td>
  </tr>
</table>

Table 36: Detailed results of on TRACE with SeqFT (start from LLAMA 2-CHAT-13B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>54.2</td>
    <td>52.2</td>
    <td>54.7</td>
    <td>55.2</td>
    <td>55.3</td>
    <td>54.3</td>
    <td>54.6</td>
    <td>55.5</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>73.4</td>
    <td>56.7</td>
    <td>54.6</td>
    <td>38.3</td>
    <td>43.1</td>
    <td>41.9</td>
    <td>50.2</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>48.9</td>
    <td>44.4</td>
    <td>44.1</td>
    <td>45.5</td>
    <td>45.9</td>
    <td>43.6</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>58.9</td>
    <td>56.3</td>
    <td>56.4</td>
    <td>56.7</td>
    <td>56.3</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>89.7</td>
    <td>84.3</td>
    <td>74.5</td>
    <td>74.6</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>54.3</td>
    <td>33.3</td>
    <td>35.8</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>64.0</td>
    <td>59.4</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.9</td>
  </tr>
  <tr>
    <th>OP</th>
    <td>54.2</td>
    <td>62.8</td>
    <td>53.4</td>
    <td>53.3</td>
    <td>56.7</td>
    <td>56.3</td>
    <td>53.0</td>
    <td>52.0</td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-8.5%</b></td>
  </tr>
</table>

Table 37: Detailed results on TRACE with SeqFT and HFT (start from LLAMA 2-CHAT-13B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>51.5</td>
    <td>47.2</td>
    <td>46.7</td>
    <td>48.1</td>
    <td>19.0</td>
    <td>47.4</td>
    <td>48.3</td>
    <td>49.2</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>70.5</td>
    <td>59.4</td>
    <td>60.2</td>
    <td>0.0</td>
    <td>60.7</td>
    <td>58.2</td>
    <td>61.2</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>52.3</td>
    <td>47.6</td>
    <td>40.5</td>
    <td>40.6</td>
    <td>43.2</td>
    <td>41.5</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>60.7</td>
    <td>60.2</td>
    <td>53.6</td>
    <td>54.6</td>
    <td>55.7</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>92.7</td>
    <td>78.5</td>
    <td>30.6</td>
    <td>60.5</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>43.7</td>
    <td>33.3</td>
    <td>33.3</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>61.7</td>
    <td>60.2</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.8</td>
  </tr>
  <tr>
    <th>OP</th>
    <td>51.5</td>
    <td>58.9</td>
    <td>52.8</td>
    <td>54.2</td>
    <td>42.5</td>
    <td>54.1</td>
    <td>47.1</td>
    <td>50.4</td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-8.9%</b></td>
  </tr>
</table>

Table 38: Detailed results on TRACE with GEM (start from LLAMA 2-CHAT-13B).
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>52.4</td>
    <td>51.5</td>
    <td>48.9</td>
    <td>49.6</td>
    <td>51.5</td>
    <td>51.0</td>
    <td>50.2</td>
    <td>51.5</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>73.4</td>
    <td>60.8</td>
    <td>61.9</td>
    <td>44.4</td>
    <td>65.3</td>
    <td>68.9</td>
    <td>67.2</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>50.2</td>
    <td>47.6</td>
    <td>41.2</td>
    <td>43.3</td>
    <td>40.9</td>
    <td>41.8</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>61.7</td>
    <td>60.1</td>
    <td>60.3</td>
    <td>58.7</td>
    <td>57.5</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>93.0</td>
    <td>88.7</td>
    <td>78.9</td>
    <td>77.7</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>44.4</td>
    <td>33.3</td>
    <td>36.7</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>61.9</td>
    <td>55.7</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>40.6</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>52.4</b></td>
    <td><b>62.5</b></td>
    <td><b>53.3</b></td>
    <td><b>55.2</b></td>
    <td><b>58.0</b></td>
    <td><b>58.8</b></td>
    <td><b>56.1</b></td>
    <td><b>53.6</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-6.1%</b></td>
  </tr>
</table>

Table 39: Detailed results on TRACE with GEM and HFT (start from LLAMA 2-CHAT-13B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>48.8</td>
    <td>51.3</td>
    <td>48.5</td>
    <td>49.3</td>
    <td>49.2</td>
    <td>47.5</td>
    <td>46.7</td>
    <td>51.4</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>62.3</td>
    <td>70.6</td>
    <td>72.4</td>
    <td>71.2</td>
    <td>71.2</td>
    <td>70.8</td>
    <td>73.0</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>44.9</td>
    <td>48.2</td>
    <td>47.4</td>
    <td>48.5</td>
    <td>47.1</td>
    <td>47.5</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>53.9</td>
    <td>55.1</td>
    <td>54.2</td>
    <td>47.5</td>
    <td>53.3</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>89.5</td>
    <td>91.6</td>
    <td>90.7</td>
    <td>89.6</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>45.7</td>
    <td>29.6</td>
    <td>30.9</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>57.5</td>
    <td>52.3</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>39.7</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>48.8</b></td>
    <td><b>56.8</b></td>
    <td><b>54.7</b></td>
    <td><b>56.0</b></td>
    <td><b>62.5</b></td>
    <td><b>59.8</b></td>
    <td><b>55.7</b></td>
    <td><b>54.7</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-0.6%</b></td>
  </tr>
</table>

Table 40: Detailed results on TRACE with Replay (start from LLAMA 2-CHAT-13B).

<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>50.2</td>
    <td>52.5</td>
    <td>53.8</td>
    <td>53.0</td>
    <td>53.4</td>
    <td>52.7</td>
    <td>52.4</td>
    <td>52.1</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>61.3</td>
    <td>74.2</td>
    <td>71.2</td>
    <td>71.8</td>
    <td>73.2</td>
    <td>72.4</td>
    <td>73.6</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>48.5</td>
    <td>48.7</td>
    <td>47.0</td>
    <td>46.9</td>
    <td>48.6</td>
    <td>47.6</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>55.7</td>
    <td>58.2</td>
    <td>55.4</td>
    <td>54.0</td>
    <td>54.5</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>83.3</td>
    <td>90.0</td>
    <td>90.1</td>
    <td>89.7</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>45.7</td>
    <td>48.1</td>
    <td>43.2</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>60.9</td>
    <td>57.5</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.0</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>50.2</b></td>
    <td><b>56.9</b></td>
    <td><b>58.8</b></td>
    <td><b>57.2</b></td>
    <td><b>62.7</b></td>
    <td><b>60.7</b></td>
    <td><b>60.9</b></td>
    <td><b>57.4</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>+1.6%</b></td>
  </tr>
</table>

Table 41: Detailed results on TRACE with Replay and HFT (start from LLAMA 2-CHAT-13B).
<table>
  <tr>
    <th>Task\Round</th>
    <th>1</th>
    <th>2</th>
    <th>3</th>
    <th>4</th>
    <th>5</th>
    <th>6</th>
    <th>7</th>
    <th>8</th>
  </tr>
  <tr>
    <td>C-STANCE</td>
    <td>52.4</td>
    <td>44.4</td>
    <td>45.1</td>
    <td>39.0</td>
    <td>0.0</td>
    <td>41.8</td>
    <td>41.1</td>
    <td>12.4</td>
  </tr>
  <tr>
    <td>FOMC</td>
    <td>-</td>
    <td>67.1</td>
    <td>58.3</td>
    <td>43.8</td>
    <td>2.2</td>
    <td>60.3</td>
    <td>57.8</td>
    <td>0.0</td>
  </tr>
  <tr>
    <td>MeetingBank</td>
    <td>-</td>
    <td>-</td>
    <td>47.3</td>
    <td>11.3</td>
    <td>18.2</td>
    <td>14.6</td>
    <td>3.2</td>
    <td>12.2</td>
  </tr>
  <tr>
    <td>Py150</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>59.2</td>
    <td>40.0</td>
    <td>47.7</td>
    <td>50.0</td>
    <td>23.6</td>
  </tr>
  <tr>
    <td>ScienceQA</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>75.4</td>
    <td>70.3</td>
    <td>71.0</td>
    <td>67.7</td>
  </tr>
  <tr>
    <td>NumGLUE-cm</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>47.5</td>
    <td>28.5</td>
    <td>25.7</td>
  </tr>
  <tr>
    <td>NumGLUE-ds</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>61.3</td>
    <td>28.6</td>
  </tr>
  <tr>
    <td>20Minuten</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>41.6</td>
  </tr>
  <tr>
    <th>OP</th>
    <td><b>52.4</b></td>
    <td><b>55.8</b></td>
    <td><b>50.2</b></td>
    <td><b>38.3</b></td>
    <td><b>27.2</b></td>
    <td><b>47.0</b></td>
    <td><b>44.7</b></td>
    <td><b>26.5</b></td>
  </tr>
  <tr>
    <th>BWT</th>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td>-</td>
    <td><b>-30.0%</b></td>
  </tr>
</table>

Table 42: Detailed results on TRACE with LoRASeqFT (start from LLAMA 2-CHAT-13B).