Preliminaries
Even though different LLMs may have varying parameter sizes and distinct vocabularies, if these models are fine-tuned on the same task, does the effect of fine-tuning exhibit similarities across them?

Given a dataset \mathcal{D} and an vanilla model \mathcal{M}^{v} , we fine-tune the model \mathcal{M}^{v} using dataset \mathcal{D} to obtain a fine-tuned model \mathcal{M}^{d} . Provided with a prompt, the models \mathcal{M}^{v} and \mathcal{M}^{d} predict the next token, resulting in \zeta_{v}\in\mathbb{R}^{|\mathcal{V}|} and \zeta_{d}\in\mathbb{R}^{|\mathcal{V}|} respectively, where \mathcal{V} denotes the size of the vocabulary. We define the effect of fine-tuning as the as the shift in logits, that is \zeta_{d}-\zeta_{v} .

When comparing the effects of fine-tuning between two distinct models, \mathcal{M}_{1}^{v} and \mathcal{M}_{2}^{v} , it involves comparing the shifts in logits when encodeing the same sequence. Specifically, given an input-response pair, denoted as [x_{1},x_{2},...,x_{n},y_{1},y_{2},...,y_{m}] , we fed this pair into the models \mathcal{M}_{1}^{v} , \mathcal{M}_{1}^{d} , \mathcal{M}_{2}^{v} , and \mathcal{M}_{2}^{d} . We then record the logits from the final layer output of each model. The logits corresponding to the response part are extracted, yielding \zeta_{1}^{v}\in\mathbb{R}^{m\times|\mathcal{V}_{1}|} , \zeta_{1}^{d}\in\mathbb{R}^{m\times|V_{1}|} , \zeta_{2}^{v}\in\mathbb{R}^{m\times|\mathcal{V}_{2}|} , and \zeta_{2}^{d}\in\mathbb{R}^{m\times|\mathcal{V}_{2}|} . To reduce the effects of varying scales in the logits from different models, we attempted to apply the LogSoftmax operation to the logits, thereby transforming them into the same logarithmic probability space.
By calculating the difference between the logits after and before fine-tuning, we obtain the fine-tuning effects.

\mathcal{T}_{\mathcal{M}1}=LogSoftmax(\zeta_{1}^{d})-LogSoftmax(\zeta_{1}^{v}) (1)

\mathcal{T}_{\mathcal{M}_{2}}=LogSoftmax(\zeta_{2}^{d})-LogSoftmax(\zeta_{2}^{v}) (2)

We selected three vanilla models, Llama2-7b , Llama2-13b , and Mistral-7B , encompassing diverse parameter scales and vocabularies. To investigate the similarity of fine-tuning effects across different models on the same dataset, we fine-tuned them individually on the GPT4-Alpaca dataset (Peng et al., 2023 ) . All models were fine-tuned using low-rank adaptation (Hu et al., 2022 ) , with hyperparameters provided in Appendix A .

To facilitate an intuitive comparison of fine-tuning effects across different models, we visualized the shifts in logits before and after fine-tuning in the form of heat maps 1 1 1 We use “What causes the northern lights?” as the input, and the output of \mathcal{M}_{1}^{d} as the response. . Taking Figure 2(a) as an example, we selected the top 100 tokens with the highest logits values in \zeta_{\textsc{Llama2-13b}}^{GPT4-Alpaca} at each time step, sorted according to the values of \mathcal{T}_{\textsc{Llama2-13b}} , and applied the corresponding indices to \mathcal{T}_{\textsc{Llama2-7b}} . We found that the upper and lower sub-figures in Figure 2 were highly similar, both exhibiting a trend of larger values on the left and smaller ones on the right, indicating a remarkably similar fine-tuning effect of different models on the same dataset. Additionally, we have quantitatively analyzed the fine-tuning effects across models using Sinkhorn divergence, as detailed in Appendix D .