TiTok: Transfer Token-level Knowledge via Contrastive Excess to Transplant LoRA

## Abstract

Abstract Large Language Models (LLMs) are widely applied in real world scenarios, but fine-tuning them comes with significant computational and storage costs.
Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA mitigate these costs, but the adapted parameters are dependent on the base model and cannot be transferred across different backbones.
One way to address this issue is through knowledge distillation, but its effectiveness inherently depends on training data.
Recent work such as TransLoRA avoids this by generating synthetic data, but this adds complexity because it requires training an additional discriminator model.
In this paper, we propose TiTok 1 1 1 Code will be released upon acceptance: https://github.com/NaughtyMaltiz16/TiTok , a new framework that enables effective LoRA T ransplantat i on through Tok en-level knowledge transfer.
Specifically, TiTok captures task-relevant information through a contrastive excess between a source model with and without LoRA.
This excess highlights informative tokens and enables selective filtering of synthetic data, all without additional models or overhead.
Through experiments on three benchmarks across multiple transfer settings, our experiments show that the proposed method is consistently effective, achieving average performance gains of + 4–8% compared to baselines overall.

## 1 Introduction

Large Language Models (LLMs) (Brown et al., 2020; Vaswani et al., 2023), have made significant progress in many real-world applications, including chatbots (OpenAI et al., 2024), search engines (Xiong et al., 2024), and coding assistants (Rozière et al., 2024).
While fine-tuning LLMs has been demonstrated to be a promising way to improve performance on downstream tasks, it incurs substantial computational and storage costs.
Parameter-Efficient Fine-Tuning (PEFT) (Houlsby et al., 2019) methods such as LoRA (Hu et al., 2021) alleviate this burden by updating only a small subset of parameters while keeping the base model frozen.
However, PEFT’s adapted parameters are dependent to the base model and cannot be transferred across different models.
This limitation is increasingly critical given the rapid release of new LLMs and the growing diversity of available models.

One considerable approach to mitigate this limitation is through Knowledge distillation (KD) (Hinton et al., 2015; Azimi et al., 2024), which transfers the knowledge embedded in a source model’s PEFT adapters to a target model with new PEFT adapters by aligning the target’s output distributions with those of the source.
However, KD is inherently data-dependent, typically requiring access to training data from target downstream tasks (Nayak et al., 2019; Liu et al., 2024), which is often unavailable or costly to obtain.
To address this limitation, TransLoRA (Wang et al., 2024) has recently proposed using synthetic data by leveraging the data synthesis capabilities of recent LLMs (Wang et al., 2023; Kim et al., 2025).
This approach enables the target model to acquire domain knowledge without direct access to the original dataset.
Nevertheless, TransLoRA requires training an additional discriminator model to filter low-quality synthetic data, which inevitably introduces extra complexity and computational overhead.
Furthermore, it primarily emphasizes the role of synthetic data, paying less attention to how the knowledge transfer process itself should be designed.

Figure: Figure 1: Overview of TiTok: Transplantation through Token-level knowledge transfer. Starting from a small set of seed prompts, the source expert model (source base model + LoRA) generates synthetic data. A contrastive excess filtering mechanism then compares the expert against its base backbone to compute token-level excess scores. Using these scores, TiTok first performs sample filtering and subsequently token selection, retaining only the most informative samples and tokens. When tokenizers differ, masks are aligned prior to training. The resulting filtered data is finally used to train a new LoRA on the target backbone, enabling efficient knowledge transfer.
Refer to caption: x1.png

Contribution.
In this paper, we propose a new framework that enables effective LoRA Transplantation through Token-level knowledge transfer (TiTok).
Our high-level idea is to selectively convey task-relevant information from the source model’s LoRA by using token-level signals to guide the transfer process, rather than relying on the entire token sequence.
We specifically capture this information through a concept we introduce as contrastive excess, obtained by comparing predictions from a source model with LoRA and the same model without LoRA.
Intuitively, this contrastive excess highlights tokens that contain important task knowledge.
This excess signal is further utilized to filter the synthetic data we generate for training, thereby enabling selective learning on samples that contain richer information.
Unlike TransLoRA, which requires training and storing an additional discriminator model for filtering, TiTok requires neither extra models nor additional training overhead.
Moreover, we design an effective mechanism to resolve tokenizer mismatches between source and target models, which enhances robustness and applicability of our framework.

We demonstrate the effectiveness of TiTok by conducting extensive experiments on three widely used benchmarks, which cover both reasoning (Big Bench Hard (Suzgun et al., 2022) and MMLU (Hendrycks et al., 2021)) and personalization tasks (LaMP (Salemi et al., 2024)).
In particular, TiTok improves the performance by +7.96% over the vanilla target model, +6.0% over KD, and +4.4% over TransLoRA, when averaged across all tasks and transfer settings.
We also explored a variety of transfer settings, including transfers within the same model, across different model families and sizes, and even across different model versions.
In every case, our approach achieves consistent improvements.
Interestingly, TiTok remains effective even when applied to external data originating from tasks different from the target task, highlighting both robustness and general applicability.
Overall, these empirical results highlight TiTok as a methodologically simple yet powerful paradigm for efficiently transferring LoRA knowledge across models in diverse scenarios.

## 2 Related Works

#### Transferring PEFT adapters.

Parameter-Efficient Fine-Tuning (PEFT) (Houlsby et al., 2019; Li & Liang, 2021) has emerged as both a practical and popular alternative to full model fine-tuning.
By requiring updates to only a small portion of parameters, it enables efficient adaptation, with LoRA (Low-Rank Adaptation) (Hu et al., 2021; Dettmers et al., 2023) standing out as one of the most widely adopted methods.
However, a fundamental limitation is that LoRA adapters are tied to the frozen backbone they were trained on, making them difficult to transfer to other base models.
To address this issue, recent studies such as TransLoRA (Wang et al., 2024) have attempted to transplant the knowledge of LoRA adapters across models by generating synthetic data (Wang et al., 2023). While effective to some extent, this approach requires an additional discriminator model to filter high-quality synthetic data, resulting in a relatively heavy pipeline. In parallel, Knowledge Distillation (KD) (Hinton et al., 2015; Azimi et al., 2024) has been widely explored as another means of knowledge transfer;
however, traditional KD typically operates within a teacher–student framework at the logit or sequence level and requires access to training data close to the original, in order to use the teacher’s distribution for supervising the student.
In contrast, our approach focuses on token-level selective transfer, offering a more fine-grained yet lightweight alternative that enables efficient transplantation of LoRA adapters across diverse models for deployment.

#### Data synthesis with LLMs.

Synthetic data generation with LLMs has attracted increasing attention as a means to reduce reliance on costly or inaccessible datasets (Wang et al., 2023; Kim et al., 2025).
Prior lines of research have leveraged synthetic data for purposes such as privacy preservation (Bu et al., 2025), data augmentation (Kumar et al., 2021), and domain adaptation (Li et al., 2023).
In our framework, synthetic data serves as a core component, enabling the transfer of LoRA adapters without access to the original training corpus, while simultaneously mitigating privacy concerns and reducing the degree of dependency on external datasets.
Moreover, since both queries and labels are generated directly by the source expert model itself, synthetic data ultimately provides a self-sufficient mechanism that fits to our objective of lightweight and effective knowledge transfer.

#### Selective token training.

Recent studies (Lin et al., 2025; Gu et al., 2020) demonstrate that not all tokens contribute equally to model training, motivating research on selective training strategies.
While such approaches have primarily been applied to accelerate optimization or reduce redundancy (Yeongbin et al., 2024; Bal et al., 2025), our work innovatively extends this concept to the setting of knowledge transfer. Specifically, our concept of excess scores (Eq. [2](https://arxiv.org/html/2510.04682v1#S3.E2)) is derived from this idea, where the contrast between the source backbone and its LoRA adapter yields token-level judgments.
This enables TiTok to transplant LoRA knowledge in a more focused and fine-grained manner, highlighting the broader applicability of selective training beyond its original scope.

## 3 TiTok : Transplanting LoRA through Token-Level Knowledge

In this section, we introduce TiTok, a framework for LoRA Transplantation through Token-level knowledge transfer (Fig. [1](https://arxiv.org/html/2510.04682v1#S1.F1)).
The core idea of TiTok is to transfer the knowledge from a source model’s LoRA adapter into a target model’s LoRA adapter by training specifically on the informative tokens within synthetic data.
Specifically, the framework consists of three components:
(1) Synthetic data generation (Sec. [3.1](https://arxiv.org/html/2510.04682v1#S3.SS1)), where the source expert model produces query–label pairs for target task;
(2) Excess score computation (Sec. [3.2](https://arxiv.org/html/2510.04682v1#S3.SS2)), which calculates token-level importance using source model; and
(3) Target model training with filtering (Sec. [3.3](https://arxiv.org/html/2510.04682v1#S3.SS3)), which trains target model with newly initialized LoRA adapter on top-ranked samples and tokens.
In addition, we propose Excess score alignment (Sec. [3.4](https://arxiv.org/html/2510.04682v1#S3.SS4)), an algortihm designed to apply TiTok even when tokenizers of the source and target models differ.
The overall algorithm of TiTok is presented in Alg. [1](https://arxiv.org/html/2510.04682v1#algorithm1).

### 3.1 Synthetic data generation via LLM prompting with few-shot data

Let $\mathcal{M}_{s}$ denote the source backbone LLM and $\mathcal{A}_{s}$ its LoRA adapter on target downstream task, which forms the source expert model $\mathcal{M}_{s}+\mathcal{A}_{s}$.
The target model, whose LoRA adapter $\mathcal{A}_{t}$ will be trained, is denoted by $\mathcal{M}_{t}$.
Then, TiTok first constructs a synthetic dataset $\mathcal{D}_{\text{s}}$, similar to the idea of TransLoRA (Wang et al., 2024).
This usage of synthetic data allows us to avoid keeping the whole original dataset for the downstream task, and simultaneously let $\mathcal{A}_{t}$ learn the knowledge encoded in the source adapter $\mathcal{A}_{s}$.
Unlike TransLoRA’s approach of using the untuned target model $\mathcal{M}_{t}$ to generate synthetic data, we use the source expert model $\mathcal{M}_{s}+\mathcal{A}_{s}$ to synthesize data (see empirical comparison in Fig. [2](https://arxiv.org/html/2510.04682v1#S4.F2)).
Concretely, $\mathcal{M}_{s}+\mathcal{A}_{s}$ synthesizes both the query and the label within a prompting-based data synthesis framework (Wang et al., 2023) (see details in Appendix [H](https://arxiv.org/html/2510.04682v1#A8));
given a few-shot data of downstream task, it first generates a query q, and then produces the corresponding label y conditioned on q.
To encourage diversity, we apply ROUGE-L filtering together with deduplication to all tasks, except for exceptional cases where such filtering is infeasible (more details are in Appendix [G](https://arxiv.org/html/2510.04682v1#A7)).
Consequently, the resulting synthetic dataset consists of query–label pairs:

$$ $\mathcal{D}_{\text{s}}=\{({\textbf{q}}_{j},{\textbf{y}}_{j})\}_{j=1}^{N}.$ (1) $$

In this way, $\mathcal{M}_{t}$ would be trained on the synthetic data $\mathcal{D}_{\text{s}}$ generated by the source expert model $\mathcal{M}_{s}+\mathcal{A}_{s}$, thereby enabling knowledge transfer without relying on the entire original dataset.

### 3.2 Contrastive excess score from source model with LoRA adapters

As described in Sec. [3.1](https://arxiv.org/html/2510.04682v1#S3.SS1), TiTok relies on synthetic data, but synthetic data is often prone to imperfections (Chen et al., 2024); thus, sophisticated filtering is essential to retain high-quality informative samples.
TransLoRA (Wang et al., 2024) tackles this challenge with a separate discriminator to filter useful queries, but this introduces the extra burden of training and maintaining an extra model.
In contrast, we propose a lightweight alternative that only utilizes the already trained source model.
Specifically, we use source model and its LoRA adapter to perform two complementary roles:
(1) the amateur role ($\mathcal{M}_{s}$) and (2) the expert role ($\mathcal{M}_{s}+\mathcal{A}_{s}$).
Then, the difference between the two roles provides an implicit supervision signal where the task information is encoded.

Formally, let $\textbf{y}=[y_{1},...,y_{L}]$ denote the synthesized response, where $y_{i}$ is a token of y corresponding to the synthesized query q.
Then, we define the excess score as:

$$ $S(y_{i})=L_{e}(y_{i})-L_{a}(y_{i}),$ (2) $$

where the amateur and expert losses on token $y_{i}$ are defined as:

$$ $L_{a}(y_{i})=\log P_{\mathcal{M}s}(y_{i}\mid\textbf{q},\textbf{y}{<i}),\quad L_{e}(y_{i})=\log P_{\mathcal{M}_{s}+\mathcal{A}s}(y_{i}\mid\textbf{q},\textbf{y}{<i}).$ (3) $$

The excess score $S(y_{i})$ quantifies the knowledge discrepancy incurred by equipment of LoRA, thereby identifying the tokens where the adapter provides a decisive contribution.
Intuitively, if the backbone model is uncertain about predicting a token but the LoRA-enhanced model assigns it with high confidence, that token will thus obtain a large excess score.
This implies that tokens with higher $S(y_{i})$ correspond to positions where the LoRA adapter injects task-specific knowledge that the backbone could not capture on its own.
In this way, the excess score functions as a fine-grained attribution signal, derived entirely from the internal behavior of the model itself, and guides training toward the specific regions of data that are most enriched with the adapter’s knowledge.

### 3.3 Target model training with sample filtering and token selection

After the computation of excess scores $S(y_{i})$, the newly initialized LoRA adapter $\mathcal{A}_{t}$ for target model $\mathcal{M}_{t}$ is trained using the synthetic samples $(\mathbf{q}_{j},\mathbf{y}_{j})\in\mathcal{D}_{\text{f}}$ with two-level filtering schemes.

First stage: Sample filtering.
We begin by filtering the synthetic dataset $\mathcal{D}_{s}$ (Eq. [1](https://arxiv.org/html/2510.04682v1#S3.E1)) at the sample level to remove less informative examples.
For each synthetic sample, we compute the mean of the excess scores $S(y_{i})$ across the tokens in $\mathbf{y}$
and retain only $M$ samples with the highest values.

$$ $\bar{S}_{j}\;=\;\frac{1}{|\mathbf{y}_{j}|}\sum_{y_{i}\in\mathbf{y}_{j}}S(y_{i}).$ (4) $$

Let $\mathcal{D}_{\mathrm{f}}$ be the set of the $M$ samples in $\mathcal{D}_{\mathrm{s}}$ with the largest $\bar{S}_{j}$:

$$ $\mathcal{D}_{\mathrm{f}}\;=\;\operatorname{Top}\!M\big\{\;(\mathbf{q}_{j},\mathbf{y}_{j})\in\mathcal{D}_{\mathrm{s}}\;:\;\bar{S}_{j}\;\big\}.$ (5) $$

Through this step, the synthetic data undergoes a filtering process, ensuring that subsequent training is concentrated specifically on the remaining examples in $\mathcal{D}_{\mathrm{f}}$ with richer knowledge signals.

Second stage: Token selection.
Next, we consider token selection; that is, $\mathcal{A}_{t}$ does not learn from all tokens within the retained samples.
Instead, it focuses only on those prioritized by the excess scores $S(y_{i})$, which are identified as most important for knowledge transfer.
To achieve this, we select the top $k\%$ of tokens ranked by their excess scores using the indicator $I_{k\%}(y_{i})$:

$$ $I_{k\%}(y_{i})=\begin{cases}1,&\text{if }\mathrm{rank}_{\mathbf{y}_{j}}\!\big(S(y_{i})\big)\leq\lfloor k\%\cdot|\mathbf{y}_{j}|\rfloor,\\ 0,&\text{otherwise},\end{cases}$ (6) $$

where $|\mathbf{y}_{j}|$ denotes the number of tokens in $\mathbf{y}_{j}$,
and $\mathrm{rank}_{\mathbf{y}_{j}}(S(y_{i}))$ indicates the rank of $S(y_{i})$
among the tokens of that response. Based on this selection, the training objective for $\mathcal{A}_{t}$ is defined as

$$ $\mathcal{L}_{\tt TiTok}=\sum_{(\mathbf{q}_{j},\mathbf{y}_{j})\in\mathcal{D}_{\text{f}}}\;\sum_{y_{i}\in\mathbf{y}_{j}}I_{k\%}(y_{i})\cdot L_{t}(y_{i}),$ (7) $$

where $L_{t}(y_{i})$ is the negative log-likelihood loss assigned by $\mathcal{M}_{t}+\mathcal{A}_{t}$ on token $y_{i}$ (only $\mathcal{A}_{t}$ is learnable). By training only on these filtered tokens (Eq. [7](https://arxiv.org/html/2510.04682v1#S3.E7)), TiTok enables $\mathcal{A}_{t}$ to efficiently acquire the source LoRA’s knowledge without access to the original training data or any external models.

### 3.4 Excess score alignment across different tokenizers

In cases of transfer between models with different tokenizers, a direct mapping of token-level signals is not possible, given that the source and target models may segment text differently.
To address this, we introduce a simple yet robust tokenizer alignment algorithm that propagates token masks (Eq. [6](https://arxiv.org/html/2510.04682v1#S3.E6)) from the source token sequence $\mathbf{y}^{(s)}$ to the target token sequence $\mathbf{y}^{(t)}$.
The algorithm first aligns token sequences using dual pointers that incrementally decode and match text spans.
Masks are then propagated using the following four rules: (1) direct copy for one-to-one mappings, (2) replication for one-to-many, (3) averaging for many-to-one, and (4) averaging with replication for many-to-many.
Finally, a top-$k\%$ selection step retains the most confident target tokens.
This process ensures consistent supervision across tokenizers, enabling reliable transfer even when models tokenize text differently. The conceptual illustration of this procedure is presented in Fig.[4](https://arxiv.org/html/2510.04682v1#A1.F4).

Figure: Algorithm 1 TiTok: Transplanting LoRA through Token-Level Knowledge

## 4 Experiment

In this section, we present our experimental results to answer the following research questions:

- •
RQ1: Can TiTok efficiently transfer knowledge of LoRA in various scenarios? (Table [1](https://arxiv.org/html/2510.04682v1#S4.T1))
- •
RQ2: What is the contribution of each component in TiTok? (Table [2](https://arxiv.org/html/2510.04682v1#S4.T2))
- •
RQ3: How sensitive is token-level selective transfer to the selection ratio? (Figure [3](https://arxiv.org/html/2510.04682v1#S4.F3))
- •
RQ4: How does the choice of model to synthesize query affect the performance? (Figure [2](https://arxiv.org/html/2510.04682v1#S4.F2))
- •
RQ5: Can TiTok transfer knowledge using data from a different or unrelated domain? (Table [3](https://arxiv.org/html/2510.04682v1#S4.T3))

### 4.1 Experimental setups

#### Models.

We mainly present our knowledge transfer experiments using models from the Mistral and Llama families.
Specifically, we designed the following LoRA transfer (*source* $\rightarrow$ *target*) setups.
(1) Mistral-7B-Inst-v0.3(^2^22[https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3)) → Mistral-7B-Inst-v0.3: the basic transfer setup,
(2) Mistral-7B-Inst-v0.3 → Llama-3.1-8B-Inst(^3^33[https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct)): the different-family model transfer setup, (3) Llama-3.2-3B-Inst(^4^44[https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct)) → Llama-3.1-8B-Inst: the different-size model transfer setup, and (4) Llama-2-7b-chat-hf(^5^55[https://huggingface.co/meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)) → Llama-3.1-8B-Inst: the different-version model transfer setup.
These setups are intended to test whether a smaller model can effectively transfer knowledge to a larger one, and to explore whether a relatively weaker model can still influence a newer, stronger model.
These various setups realistically reflect how LLMs develop today, where various models are released and newer, improved versions keep emerging.
Model names are abbreviated for clarity and are used consistently in all tables and figures:
1) “Mistral 7B” = Mistral-7B-Instruct-v0.3,
2) “Llama3 8B” = Llama-3.1-8B-Instruct,
3) “Llama3 3B” = Llama-3.2-3B-Instruct,
4) “Llama2 7B” = Llama-2-7b-chat-hf.

#### Baselines.

To demonstrate the effectiveness of TiTok, we compare it against three baselines: (i) Vanilla, the target base model without any fine-tuning;
(ii) KD(+MinED), knowledge distillation (KD) from source expert model (Hinton et al., 2015).
When the source and target models use different tokenizers, the original KD is not applicable.
In this case, we use Minimum-Edit-Distance (MinED) tokenizer alignment (Wan et al., 2024), which aligns both token sequences and vocabulary distributions via dynamic-programming sequence alignment for near matches (e.g., “gets” $\leftrightarrow$ “get”) and by mapping probability mass to nearest edit-distance neighbors (e.g., “immediately” $\leftrightarrow$ “immediate”).
We use the synthesized data by TransLoRA as the training data for KD and MinED
; and (iii) TransLoRA (Wang et al., 2024), a prior method in which the vanilla target model synthesizes the queries, while the source model with its LoRA adapter generates the corresponding synthetic labels. A discriminator is then employed to filter the synthetic data for training the target LoRA adapter.

#### Datasets.

Following prior work in TransLoRA, we first conduct experiments on two representative benchmarks: (1) Big-Bench Hard (BBH) (Suzgun et al., 2022) and (2) Massive Multitask Language Understanding (MMLU) (Hendrycks et al., 2021).
BBH consists of 27 challenging reasoning tasks structured as multiple choice or short answer questions, designed to test compositional generalization and advanced problem solving abilities of a Language Model.
Meanwhile, MMLU covers 57 tasks across diverse academic subjects, presented in multiple choice format to evaluate broad knowledge and reasoning skills of a model.
Since both benchmarks only provide test sets, we split the data into 90% for training the source expert model $\mathcal{M}_{s}+\mathcal{A}_{s}$ and the remaining 10% for evaluation.

To extend our approach to personalization and text generation, we additionally conduct experiments on LaMP benchmark (Salemi et al., 2024), focusing exclusively on its generation tasks.
In particular, we experiment with (3) News Headline Generation (LaMP 4) and (4) Scholarly Title Generation (LaMP 5), as they are the only text generation tasks that are both accessible and reliably evaluable.
The remaining LaMP tasks are excluded, given that LaMP 1-3 are discriminative, and LaMP 6-7 lack gold labels.
For LaMP tasks, the source expert model $\mathcal{M}_{s}+\mathcal{A}_{s}$ is trained on data from the 30 users with the longest activity histories.
From each user, we use 200 data points for training and 50 data points for validation, a design choice intended to conduct a more rigorous and robust evaluation.
In total, each LaMP task contains 6,000 training examples and 1,500 evaluation examples.

To assess performance on the BBH and MMLU, we measure the average accuracy using the LM-Eval Harness (Gao et al., 2024).
Following the setup used in TransLoRA (Wang et al., 2024), all tasks are conducted in a zero-shot setting.
Meanwhile, for the LaMP tasks, we adopt ROUGE-1 and ROUGE-L scores as evaluation metrics, following the benchmark’s primary evaluation metrics.

#### Implementation details.

For training of both source and target models, we use a learning rate of $5\times 10^{-5}$, train for 2 epochs with a batch size of 4, and apply LoRA with rank $r=8$, scaling factor $\alpha=8$, and dropout $0.05$.
Optimization is performed using AdamW with weight decay $1\times 10^{-2}$, together with a linear learning rate schedule and a warmup ratio of $0.1$.
For synthetic data generation, we provide five samples from the original training data as few-shot exemplars, and apply top-p sampling (Holtzman et al., 2020) to generate both queries and labels, with sampling hyperparameters tuned individually for each task.
To further encourage diversity, we apply ROUGE-L filtering with a threshold of $0.7$ and deduplication to remove redundant queries, following Wang et al. (2023).
For tasks where ROUGE-based filtering is infeasible, we apply only deduplication (see Appendix [G](https://arxiv.org/html/2510.04682v1#A7)).
For the initial synthetic pool, we generate $2M$ synthetic samples and, after filtering with contrastive excess scores, retain the top $M$, where $M$ equals the source training set size (see Appendix [F](https://arxiv.org/html/2510.04682v1#A6)).
For token selection, the selection ratio $k\%$ is fixed at $70\%$ across all tasks and transfer settings, except for the Llama3 3B $\rightarrow$ Llama3 8B transfer setting, where we find that $k\%=30\%$ consistently yields the best performance.
For inference, we adopt greedy decoding to ensure deterministic evaluation.

### 4.2 Main results

**Table 1: Main results. Experiments on BBH, MMLU, News Headline and Scholarly Title Generation tasks under four transfer settings. BBH and MMLU are reasoning tasks and are evaluated using LM-Eval Harness, while News Headline and Scholarly Title Generation represent personalization tasks and are evaluated with ROUGE-1/L. All evaluations are zero-shot. Best scores are in bold.**
| Transfer | Method | BBH | MMLU | News Headline | Scholarly Title |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Acc. | Acc. | ROUGE-1 | ROUGE-L | ROUGE-1 | ROUGE-L |  |  |
| Mistral 7B<br>$\rightarrow$ Mistral 7B | Vanilla | 0.397 | 0.557 | 0.117 | 0.101 | 0.381 | 0.311 |
| KD | 0.426 | 0.556 | 0.121 | 0.107 | 0.392 | 0.320 |  |
| TransLoRA | 0.424 | 0.533 | 0.155 | 0.136 | 0.447 | 0.382 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.432 | \cellcolorgray!150.563 | \cellcolorgray!150.160 | \cellcolorgray!150.142 | \cellcolorgray!150.473 | \cellcolorgray!150.414 |  |
| Mistral 7B<br>$\rightarrow$ Llama3 8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD+MinED | 0.477 | 0.484 | 0.127 | 0.112 | 0.455 | 0.389 |  |
| TransLoRA | 0.471 | 0.472 | 0.122 | 0.108 | 0.461 | 0.397 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.482 | \cellcolorgray!150.488 | \cellcolorgray!150.140 | \cellcolorgray!150.124 | \cellcolorgray!150.464 | \cellcolorgray!150.403 |  |
| Llama3 3B<br>$\rightarrow$ Llama3 8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD | 0.470 | 0.475 | 0.126 | 0.111 | 0.449 | 0.383 |  |
| TransLoRA | 0.460 | 0.466 | 0.121 | 0.107 | 0.455 | 0.387 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.509 | \cellcolorgray!150.475 | \cellcolorgray!150.127 | \cellcolorgray!150.113 | \cellcolorgray!150.457 | \cellcolorgray!150.392 |  |
| Llama2 7B<br>$\rightarrow$ Llama3 8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD+MinED | 0.473 | 0.478 | 0.125 | 0.111 | 0.450 | 0.384 |  |
| TransLoRA | 0.472 | 0.468 | 0.123 | 0.109 | 0.453 | 0.388 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.510 | \cellcolorgray!150.479 | \cellcolorgray!150.140 | \cellcolorgray!150.122 | \cellcolorgray!150.461 | \cellcolorgray!150.404 |  |

Table [1](https://arxiv.org/html/2510.04682v1#S4.T1) summarizes the experimental results on BBH, MMLU, and LaMP (News Headline and Scholarly Title generation) across four transfer settings.
First, we observe that transfer within the same model family is highly effective; when transplanting the LoRA adapter trained on the Mistral 7B into a fresh instance of the same model, TiTok consistently surpasses all baselines.
In particular, TiTok improves the vanilla model by 24.08% on average across all tasks, and further outperforms the KD and TransLoRA baselines by 19.63% and 4.91%, respectively.
Taken together, the findings show that, as a first step, transfer within the same family is reliably successful.

Beyond same family transfer, we also find that TiTok is highly effective in cross-model transfer settings.
For example, when transferring from Mistral 7B to Llama 8B, TiTok delivers an average improvement across all tasks of 7.11% over the vanilla model, while outperforming KD by 4.73% and TransLoRA by 6.24%. This highlights that TiTok is not confined to intra-family knowledge transfer, but can successfully bridge across architectures. In the case of Llama3 3B to Llama3 8B, TiTok yields average gains of 3.45% against vanilla, 2.49% against KD, and 4.14% against TransLoRA. These results suggest that TiTok scales effectively with model size, making it useful when moving from lightweight to larger models without losing efficiency. Finally, when transferring from Llama2 7B to Llama3 8B, TiTok performs average advantages of 7.43% over vanilla, 6.28% over KD, and 7.22% over TransLoRA. This indicates that TiTok adapts robustly even across different model versions, implying practical relevance when models are upgraded in real-world pipelines.

Collectively, these results provide strong evidence that TiTok not only excels in transfers within the same model family, but also demonstrates broad effectiveness in cross-model transfers, thereby highlighting its robustness across a wide spectrum of model scales, versions, and families.

**Table 2: Ablation study. ”Sample filtering” uses the mean token-level contrastive excess score to remove uninformative examples, while “Token selection” further refines the retained data by selectively keeping only the most informative tokens within each sample. Without sample filtering, data are randomly sampled. Results are reported as accuracy (Acc.) for BBH and MMLU, averaged across tasks. Meanwhile, we report ROUGE-1 (R-1) and ROUGE-L (R-L) for LaMP tasks.**
| Settings | BBH | MMLU | News Headline | Scholarly Title |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Sample filtering | Token selection | Acc. | Acc. | R-1 | R-L | R-1 | R-L |
| ✗ | ✗ | 0.458 | 0.485 | 0.133 | 0.117 | 0.456 | 0.393 |
| ✗ | ✓ | 0.463 | 0.496 | 0.137 | 0.121 | 0.460 | 0.397 |
| ✓ | ✗ | 0.470 | 0.500 | 0.139 | 0.122 | 0.460 | 0.397 |
| ✓ | ✓ | 0.483 | 0.501 | 0.142 | 0.125 | 0.464 | 0.403 |

### 4.3 Additional analyses

#### Ablation study.

To validate the contribution of each component in our framework, we conduct additional experiments by selectively excluding the sample filtering and token selection mechanisms in Sec. [3.3](https://arxiv.org/html/2510.04682v1#S3.SS3).
We report the average performance scores across all four transfer settings, and the results are presented in Table [2](https://arxiv.org/html/2510.04682v1#S4.T2).
When sample filtering is not applied (1st and 2nd rows), the training data for the target model is randomly sampled from the full synthetic dataset.
Under this setting, applying only token selection (2nd row) performs noticeably better than the purely random baseline.
This demonstrates the effectiveness of our token selection approach and empirically confirms that contrastive excess reliably identifies and selects the most informative tokens.
Similarly, when only sample filtering is applied (1st and 3rd rows), the results improve over pure random sampling, indicating that selecting high-quality data is essential.
This finding further underscores that incorporating an effective filtering mechanism for synthetic data is essential. In our framework, contrastive excess serves this role by reliably selecting samples with richer and more informative signals, as demonstrated clearly by the empirical results
Finally, the results show that combining both stages (4th row) achieves the best performance, confirming their complementary roles in filtering high-quality examples and selecting informative tokens for effective knowledge transfer.

Figure: Figure 2: Impact of query source. Across transfer settings, using the source expert model to synthesize query generally yields better performance. The accuracy averaged over all BBH tasks are reported.
Refer to caption: x2.png

#### Impact of query generation model.

We now move on to examine how the choice of query generation model influences performance.
Practically, synthetic queries can be generated either by the source model or by the target model, with the latter being the approach originally adopted in TransLoRA’s pipeline (Wang et al., 2024).
Figure [2](https://arxiv.org/html/2510.04682v1#S4.F2) compares these two options across different transfer settings, with the reported scores averaged over all BBH tasks.
Notably, we observe that using the source expert model (source backbone + LoRA) for query generation generally yields stronger performance than using the target model.
One possible explanation is that when both the synthetic queries and the corresponding labels are generated by the same model, they remain closer to its training distribution.
This alignment between queries and labels likely makes the supervision more coherent and accurate, thereby facilitating more effective transfer.
These results suggest that maintaining distributional alignment between queries and labels is a key factor for improving knowledge transfer, thereby providing empirical justification for our design choice of using the source model as both the synthetic query and label generator.

Figure: (a) (a) BBH (accuracy) Mistral 7B $\to$ Mistral 7B
Refer to caption: x3.png

#### Effect of token selection ratio.

We now proceed to explore the impact of token selection ratios ($k\%$) in our framework.
Fig. [3](https://arxiv.org/html/2510.04682v1#S4.F3) presents three representative curves: one task (BBH) under two transfer settings, and another task (News Headline Generation) under the same transfer setting for comparison.
For BBH with the same backbone on source and target (Mistral 7B→Mistral 7B), a moderate token selection ratio of 40-70% yields the best results.
Very low ratios (10-20%) lead to underfitting, while a 100% ratio is ineffective as it applies no filtering at all.
Interestingly, in a weak-to-strong transfer (Llama2 7B → Llama3 8B) for the same BBH task, performance improves as k% decreases.
This suggests that selecting only the tokens with the highest excess loss effectively filters out the majority of noisy regions where the weaker model is uncertain.
By discarding the majority of tokens where the weaker model lacks confidence, TiTok can significantly reduce negative transfer.
This trend, however, reverses for the News Headline Generation task under the same weak-to-strong transfer setting.
For this task, a larger k% is generally better, potentially because even a weaker model can provide valuable lexical and stylistic cues that are useful for personalization.
Consequently, unlike reasoning-focused tasks such as BBH, which are sensitive to noisy supervision, personalization tasks like News Headline Generation benefit more broadly from the source model’s outputs, even when the source model is relatively weaker than the target model.

#### Effectiveness of transfer through external data source.

**Table 3: Transfer using external data. ROUGE-1 (R-1) and ROUGE-L (R-L) on News Headline and Scholarly Title Generation under three data settings for Mistral 7B→Mistral 7B: (1) other-user, (2) mixed-user, (3) cross-task. Best scores are in bold.**
| Data Setting | Method | News Headline | Scholarly Title |  |  |
| --- | --- | --- | --- | --- | --- |
| R-1 | R-L | R-1 | R-L |  |  |
| Other-user | Vanilla | 0.117 | 0.101 | 0.381 | 0.311 |
| KD | 0.127 | 0.113 | 0.405 | 0.333 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.151 | \cellcolorgray!150.136 | \cellcolorgray!150.481 | \cellcolorgray!150.425 |  |
| Mixed-user | Vanilla | 0.117 | 0.101 | 0.381 | 0.311 |
| MinED | 0.124 | 0.110 | 0.409 | 0.337 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.151 | \cellcolorgray!150.137 | \cellcolorgray!150.480 | \cellcolorgray!150.422 |  |
| Cross-task | Vanilla | 0.117 | 0.101 | 0.381 | 0.311 |
| MinED | 0.118 | 0.106 | 0.403 | 0.331 |  |
| \cellcolorgray!15TiTok (ours) | \cellcolorgray!150.133 | \cellcolorgray!150.120 | \cellcolorgray!150.450 | \cellcolorgray!150.383 |  |

We further examine whether TiTok can transfer knowledge effectively in cases where synthetic data is not preferred, and thus external data is used as an alternative.
To this end, we evaluate three alternative settings on LaMP tasks for Mistral 7B→Mistral 7B transfer setup: (1) using data from a randomly chosen different user, (2) mixing data from multiple users, and (3) transferring across tasks, where data of Scholarly Title Generation is used to train on News Headline Generation and vice versa (i.e., out-of-distribution scenario).
The results are presented in Table [3](https://arxiv.org/html/2510.04682v1#S4.T3).
Remarkably, TiTok consistently outperforms all baselines across these heterogeneous external settings.
These findings demonstrate that TiTok is not restricted to synthetic data scenarios, but can also adapt effectively under external or user-provided data conditions.
This highlights the flexibility of TiTok for practical deployment in diverse real-world application scenarios and further underscores its adaptability across different data conditions, as it is effective even when external data is used as an alternative to synthetic data.

## 5 Conclusion

In this paper, we propose TiTok, an efficient framework that transfers LoRA knowledge from a source model to a target model by training only on a selectively chosen set of highly informative tokens.
At its core, TiTok leverages token-level signals to distill task-relevant information from the source adapter, rather than relying on the entire token sequence.
By focusing supervision on the regions where the adapter contributes most, TiTok achieves stronger and more targeted knowledge transfer.
With this simple yet effective design, TiTok proves robust across tasks and consistently surpasses existing baselines, making it a practical solution for efficient knowledge transfer.

#### Limitations and Future Directions

While TiTok has shown robustness and clear advantages over existing methods, there remain opportunities to refine and extend the framework.
Firstly, although TiTok is designed to minimize dependence on original data, it still requires a small number of seed examples to generate synthetic data via prompting.
Nevertheless, this reliance is modest, as we avoid using full datasets.
In addition, our experiments confirm that external data can also serve as an effective alternative, reinforcing the flexibility of the approach.
Regarding token selection, TiTok currently applies a fixed threshold to determine which tokens are retained.
While this simple design is effective and stable across tasks, future work could explore more adaptive or data-driven thresholding strategies to further enhance efficiency without compromising robustness.

## Ethics Statement

We conduct this research in full accordance with established ethical standards.
For our experiments we rely exclusively on publicly available datasets such as LaMP, MMLU, and BBH and use them strictly in accordance with their intended purpose for academic research.
For the LaMP tasks, which involve user data, our TiTok framework aligns with ethical considerations by minimizing data dependence.
It does not store or expose raw user data and only updates a small set of task and user-specific parameters.
This helps safeguard privacy while enabling efficient knowledge transfer.

## Reproducibility Statement

We provide a comprehensive description of our implementation in Section [4](https://arxiv.org/html/2510.04682v1#S4), including pipeline configurations, hyperparameters, models, datasets, and evaluation metrics.
The source code for our implementation and experiments will be made publicly available in a repository upon publication.

## References

- Azimi et al. (2024)
Rambod Azimi, Rishav Rishav, Marek Teichmann, and Samira Ebrahimi Kahou.
Kd-lora: A hybrid approach to efficient fine-tuning with lora and knowledge distillation.
*arXiv preprint arXiv:2410.20777*, 2024.
- Bal et al. (2025)
Melis Ilayda Bal, Volkan Cevher, and Michael Muehlebach.
Eslm: Risk-averse selective language modeling for efficient pretraining, 2025.
URL [https://arxiv.org/abs/2505.19893](https://arxiv.org/abs/2505.19893).
- Brown et al. (2020)
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei.
Language models are few-shot learners, 2020.
URL [https://arxiv.org/abs/2005.14165](https://arxiv.org/abs/2005.14165).
- Bu et al. (2025)
Hyungjune Bu, Chanjoo Jung, Minjae Kang, and Jaehyung Kim.
Personalized llm decoding via contrasting personal preference, 2025.
URL [https://arxiv.org/abs/2506.12109](https://arxiv.org/abs/2506.12109).
- Chen et al. (2024)
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al.
Alpagasus: Training a better alpaca with fewer data.
In *International Conference on Learning Representations (ICLR)*, 2024.
- Dettmers et al. (2023)
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.
Qlora: Efficient finetuning of quantized llms.
In *Advances in Neural Information Processing Systems (NeurIPS)*, 2023.
- Gao et al. (2024)
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou.
The language model evaluation harness, 07 2024.
URL [https://zenodo.org/records/12608602](https://zenodo.org/records/12608602).
- Gu et al. (2020)
Yuxian Gu, Zhengyan Zhang, Xiaozhi Wang, Zhiyuan Liu, and Maosong Sun.
Train no evil: Selective masking for task-guided pre-training.
In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), *Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)*, pp. 6966–6974, Online, November 2020. Association for Computational Linguistics.
doi: 10.18653/v1/2020.emnlp-main.566.
URL [https://aclanthology.org/2020.emnlp-main.566/](https://aclanthology.org/2020.emnlp-main.566/).
- Hendrycks et al. (2021)
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt.
Measuring massive multitask language understanding, 2021.
URL [https://arxiv.org/abs/2009.03300](https://arxiv.org/abs/2009.03300).
- Hinton et al. (2015)
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.
Distilling the knowledge in a neural network.
*arXiv preprint arXiv:1503.02531*, 2015.
- Holtzman et al. (2020)
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi.
The curious case of neural text degeneration.
In *International Conference on Learning Representations (ICLR)*, 2020.
- Houlsby et al. (2019)
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.
Parameter-efficient transfer learning for nlp.
In *International conference on machine learning*, pp. 2790–2799. PMLR, 2019.
- Hu et al. (2021)
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.
Lora: Low-rank adaptation of large language models, 2021.
URL [https://arxiv.org/abs/2106.09685](https://arxiv.org/abs/2106.09685).
- Kim et al. (2025)
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig.
Evaluating language models as synthetic data generators.
In *Annual Meeting of the Association for Computational Linguistics (ACL)*, 2025.
- Kumar et al. (2021)
Varun Kumar, Ashutosh Choudhary, and Eunah Cho.
Data augmentation using pre-trained transformer models, 2021.
URL [https://arxiv.org/abs/2003.02245](https://arxiv.org/abs/2003.02245).
- Li & Liang (2021)
Xiang Lisa Li and Percy Liang.
Prefix-tuning: Optimizing continuous prompts for generation.
*arXiv preprint arXiv:2101.00190*, 2021.
- Li et al. (2023)
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin.
Synthetic data generation with large language models for text classification: Potential and limitations, 2023.
URL [https://arxiv.org/abs/2310.07849](https://arxiv.org/abs/2310.07849).
- Lin et al. (2025)
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, and Weizhu Chen.
Rho-1: Not all tokens are what you need, 2025.
URL [https://arxiv.org/abs/2404.07965](https://arxiv.org/abs/2404.07965).
- Liu et al. (2024)
He Liu, Yikai Wang, Huaping Liu, Fuchun Sun, and Anbang Yao.
Small scale data-free knowledge distillation.
In *Conference on Computer Vision and Pattern Recognition (CVPR)*, 2024.
- Nayak et al. (2019)
Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty.
Zero-shot knowledge distillation in deep networks.
In *Proceedings of the International Conference on Machine Learning (ICML)*, 2019.
- OpenAI et al. (2024)
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapiro, Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo
Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Łukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan
Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O’Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez,
Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia
Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph.
Gpt-4 technical report, 2024.
URL [https://arxiv.org/abs/2303.08774](https://arxiv.org/abs/2303.08774).
- Rozière et al. (2024)
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve.
Code llama: Open foundation models for code, 2024.
URL [https://arxiv.org/abs/2308.12950](https://arxiv.org/abs/2308.12950).
- Salemi et al. (2024)
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani.
Lamp: When large language models meet personalization, 2024.
URL [https://arxiv.org/abs/2304.11406](https://arxiv.org/abs/2304.11406).
- Suzgun et al. (2022)
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Le, Ed H. Chi, Denny Zhou, and Jason Wei.
Challenging big-bench tasks and whether chain-of-thought can solve them, 2022.
URL [https://arxiv.org/abs/2210.09261](https://arxiv.org/abs/2210.09261).
- Vaswani et al. (2023)
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin.
Attention is all you need, 2023.
URL [https://arxiv.org/abs/1706.03762](https://arxiv.org/abs/1706.03762).
- Wan et al. (2024)
Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi.
Knowledge fusion of large language models, 2024.
URL [https://arxiv.org/abs/2401.10491](https://arxiv.org/abs/2401.10491).
- Wang et al. (2024)
Runqian Wang, Soumya Ghosh, David Cox, Diego Antognini, Aude Oliva, Rogerio Feris, and Leonid Karlinsky.
Trans-lora: towards data-free transferable parameter efficient finetuning.
In *Advances in Neural Information Processing Systems (NeurIPS)*, 2024.
- Wang et al. (2023)
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi.
Self-instruct: Aligning language models with self-generated instructions.
In *Annual Meeting of the Association for Computational Linguistics (ACL)*, 2023.
- Xiong et al. (2024)
Haoyi Xiong, Jiang Bian, Yuchen Li, Xuhong Li, Mengnan Du, Shuaiqiang Wang, Dawei Yin, and Sumi Helal.
When search engine services meet large language models: Visions and challenges, 2024.
URL [https://arxiv.org/abs/2407.00128](https://arxiv.org/abs/2407.00128).
- Yeongbin et al. (2024)
Seo Yeongbin, Dongha Lee, and Jinyoung Yeo.
Train-attention: Meta-learning where to focus in continual knowledge learning.
*Advances in Neural Information Processing Systems*, 37:58284–58308, 2024.

## Appendix A Tokenizer Alignment Algorithm

Figure: Figure 4: Overview of TiTok’s tokenizer alignment algorithm. The algorithm handles cases where the source and target models use different tokenizers. The binary mask scores assigned by the source model are averaged within aligned spans and propagated to target tokens, producing fractional scores that guide top-$k\%$ token selection for training the target model’s LoRA adapter.
Refer to caption: x6.png

When the source and target models use different tokenizers, direct token-level transfer is not possible.
To address this, we implement a dual-pointer alignment procedure.
The algorithm maintains two pointers, one for the source tokens and one for the target tokens.
At each step, the source pointer advances by one token, accumulating a decoded segment, while the target pointer incrementally extends its own segment until the normalized texts match.
Once a match is found, the corresponding spans are recorded as an alignment, and the target pointer jumps forward to that position.
After alignment, we apply masking rules to propagate the source binary mask scores (values that indicate whether a token should be kept or discarded) to the target tokens.
Specifically:

- 1)
One-to-One: binary mask score is directly copied.
- 2)
One-to-Many: score is replicated across all aligned target tokens.
- 3)
Many-to-One: averaged scores of multiple source tokens are assigned to the target token.
- 4)
Many-to-Many: the averaged score of the source tokens is assigned to the corresponding aligned target tokens.

This process yields fractional mask scores that capture the relative importance of each target token.
Finally, we keep only the top-$k\%$ of tokens according to these scores, producing a final binary selection mask over the target tokens that preserves the most informative regions while discarding less relevant ones.
An overview of this tokenizer alignnment algorithm is presented in Figure [4](https://arxiv.org/html/2510.04682v1#A1.F4).

## Appendix B Details of Datasets

### B.1 Big-Bench Hard (BBH)

Big-Bench Hard (BBH) (Suzgun et al., 2022) is designed as a rigorous benchmark for evaluating model performance on challenging reasoning problems, including multi-step logical reasoning, symbolic manipulation, and commonsense inference. The tasks are formatted as multiple choice questions. Since BBH is originally a test-only benchmark, we split 90% of the data for training the source expert model and reserved 10% for evaluation. The 27 BBH tasks are categorized in Table [4](https://arxiv.org/html/2510.04682v1#A2.T4).

**Table 4: Categorization of the 27 Big-Bench Hard (BBH) tasks.**
| Category | Tasks |
| --- | --- |
| Logical Reasoning | boolean_expressions, causal_judgement, date_understanding, disambiguation_qa,<br>dyck_languages, formal_fallacies, logical_deduction_three_objects,<br>logical_deduction_five_objects, logical_deduction_seven_objects,<br>temporal_sequences, tracking_shuffled_objects_three_objects,<br>tracking_shuffled_objects_five_objects, tracking_shuffled_objects_seven_objects, web_of_lies |
| Linguistic | hyperbaton, ruin_names, salient_translation_error_detection, snarks, word_sorting |
| Mathematical / Symbolic | geometric_shapes, multistep_arithmetic_two, object_counting, reasoning_about_colored_objects |
| Applied / Knowledge | movie_recommendation, navigate, penguins_in_a_table, sports_understanding |

**Table 5: Categorization of the 57 MMLU tasks.**
| Category | Tasks |
| --- | --- |
| STEM (29) | abstract_algebra, anatomy, astronomy, college_biology, college_chemistry,<br>college_computer_science, college_mathematics, college_medicine, college_physics,<br>computer_security, conceptual_physics, electrical_engineering, elementary_mathematics,<br>formal_logic, high_school_biology, high_school_chemistry, high_school_computer_science,<br>high_school_mathematics, high_school_physics, high_school_statistics, machine_learning,<br>medical_genetics, nutrition, professional_medicine, professional_psychology, virology,<br>clinical_knowledge, human_aging, human_sexuality |
| Humanities (16) | business_ethics, formal_fallacies, jurisprudence, logical_fallacies, philosophy, prehistory,<br>world_religions, moral_disputes, moral_scenarios, professional_law, professional_accounting,<br>high_school_world_history, high_school_us_history, high_school_european_history,<br>global_facts, security_studies |
| Social Sciences (10) | econometrics, high_school_macroeconomics, high_school_microeconomics, management, marketing,<br>public_relations, sociology, us_foreign_policy, high_school_geography,<br>high_school_government_and_politics |
| Other (2) | miscellaneous, global_facts |

### B.2 Massive Multitask Language Understanding (MMLU)

Massive Multitask Language Understanding (MMLU) (Hendrycks et al., 2021) is a comprehensive benchmark for evaluating model performance across a broad range of knowledge intensive tasks. The benchmark consists of multiple choice questions and, similarly to BBH, we also apply a 90%/10% split of the original test-only data. All the 57 subtasks are categorized in Table [5](https://arxiv.org/html/2510.04682v1#A2.T5).

### B.3 LaMP tasks

We utilize the LaMP benchmark to evaluate whether TiTok is also effective in the personalization setting.
Among the LaMP tasks, we focus on the two text generation tasks that are suitable, accessible, and evaluable in our setting:

- •
News Headline Generation (LaMP 4). Given an author profile consisting of previously written headlines, the model is asked to generate a headline for a new news article. The task evaluates if the model can adapt its output to reflect the author’s characteristic style in journalistic writing.
- •
Scholarly Title Generation (LaMP 5). Using an author profile built from prior titles of academic publications, the model generates a title for a new given abstract. The task assesses the model’s ability to capture and reproduce distinctive conventions of academic writing.

## Appendix C Baseline Details

To evaluate the effectiveness of TiTok, we compare against several baselines as follows:

### C.1 Vanilla

The vanilla baseline corresponds to the standard target base model without any additional training or knowledge transfer from the source model. This setup reflects the performance of the target model in its raw initialized state and serves as a lower bound for evaluating transfer methods. Intuitively, achieving performance that surpasses this baseline provides clear evidence that the target model has successfully acquired and internalized knowledge transferred from the source model.

### C.2 Knowledge Distillation (KD) (+MinED)

The knowledge distillation (KD) (Hinton et al., 2015; Azimi et al., 2024) baseline trains the student model to mimic the teacher model’s output distribution. Specifically, the loss is a weighted sum of the cross entropy objective and the KL divergence between the teacher and student distributions. In our setup, the KD experiments are conducted using the TransLoRA filtered synthetic datasets.

For cases where the source and target models use mismatched tokenizers when performing KD, we apply the Minimal Edit Distance (MinED) alignment method (Wan et al., 2024). In particular, MinED matches tokens across different vocabularies by minimizing the number of character level edits. For example, it can align “get” with “gets,” “color” with “colour,” or “analysis” with “analyses.” This tokenizer alignment approach avoids degeneracy issues in token alignment when conducting KD, though it is different from our dual-pointer based alignment algorithm.

### C.3 TransLoRA

The TransLoRA baseline (Wang et al., 2024) transfers LoRA knowledge by generating synthetic data. In this method, the vanilla target model synthesizes queries, and the source model with its LoRA adapter provides the corresponding labels. Subsequently, a discriminator, separately trained with the source model as its base, is applied to filter the synthetic data used for fine-tuning the target model’s LoRA adapter. For the sake of consistency and fair comparison,we adopt the same hyperparameter settings from our experiments when applying the TransLoRA baseline.

## Appendix D Usage of AI Assistants

AI assistants are minimally used in this work, restricted solely to language refinement. such as grammar correction, punctuation, and sentence structure. All research ideas, methodologies, and analyses are the original contributions of the authors. Thus, the use of AI is confined to editorial support and did not influence the originality or intellectual contributions of the work.

## Appendix E Comparison with Additional Baseline

Here, we introduce an additional baseline that leverages few-shot supervision.
In practice, our synthetic data generation process is seeded with a small set of prompts, which can be regarded as few-shot data.
To be thorough, we therefore establish a baseline trained only on these five seed prompts, referred to as KD (5-shot), so that our evaluation of TiTok also considers minimal few-shot supervision.
The results of this additional baseline are presented in Table [6](https://arxiv.org/html/2510.04682v1#A5.T6).

Across all transfer settings, KD on only 5-shot samples provides only marginal improvements over the vanilla model, with average gains of 0.9% on reasoning tasks (BBH and MMLU) and 0.6–1.2% on personalization tasks (News Headline and Scholarly Title Generation).
In contrast, TiTok delivers substantial improvements over this baseline, achieving 4.7% and 2.1% average gains on reasoning tasks and 8.9–16.6% average gains on personalization tasks.
Winning over KD with only 5 few-shot samples shows that TiTok is not simply the result of a few-shot effect.
Instead, the five seed prompts act only as a starting point, which our framework expands into richer and more effective synthetic training signals.
This finding confirms that the true strength of TiTok lies in how it effectively leverages limited supervision rather than in the few-shot data itself.

**Table 6: Comparison with a few-shot KD baseline using five seed prompts. BBH and MMLU are reported as the average accuracy across tasks, while News Headline and Scholarly Title Generation tasks are evaluated using ROUGE-1 (R-1) and ROUGE-L (R-L). KD (5-shot) denotes knowledge distillation performed with only five few-shot examples. Best scores are in bold.**
| Transfer | Method | BBH | MMLU | News Headline | Scholarly Title |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | Acc. | Acc. | R-1 | R-L | R-1 | R-L |
| Mistral 7B $\to$ Mistral 7B | Vanilla | 0.397 | 0.557 | 0.117 | 0.101 | 0.381 | 0.311 |
| KD (5-shot) | 0.402 | 0.558 | 0.118 | 0.104 | 0.383 | 0.312 |  |
| \cellcolorgray!10TiTok (ours) | \cellcolorgray!100.432 | \cellcolorgray!100.563 | \cellcolorgray!100.160 | \cellcolorgray!100.142 | \cellcolorgray!100.473 | \cellcolorgray!100.414 |  |
| Mistral 7B $\to$ Llama 8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD (5-shot) | 0.470 | 0.477 | 0.126 | 0.111 | 0.446 | 0.379 |  |
| \cellcolorgray!10TiTok (ours) | \cellcolorgray!100.482 | \cellcolorgray!100.488 | \cellcolorgray!100.140 | \cellcolorgray!100.124 | \cellcolorgray!100.464 | \cellcolorgray!100.403 |  |
| Llama3B $\to$ Llama8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD (5-shot) | 0.470 | 0.479 | 0.126 | 0.111 | 0.446 | 0.379 |  |
| \cellcolorgray!10TiTok (ours) | \cellcolorgray!100.509 | \cellcolorgray!100.475 | \cellcolorgray!100.127 | \cellcolorgray!100.113 | \cellcolorgray!100.457 | \cellcolorgray!100.392 |  |
| Llama2 7B $\to$ Llama3 8B | Vanilla | 0.469 | 0.469 | 0.125 | 0.110 | 0.444 | 0.378 |
| KD (5-shot) | 0.470 | 0.477 | 0.126 | 0.111 | 0.446 | 0.380 |  |
| \cellcolorgray!10TiTok (ours) | \cellcolorgray!100.510 | \cellcolorgray!100.479 | \cellcolorgray!100.140 | \cellcolorgray!100.122 | \cellcolorgray!100.461 | \cellcolorgray!100.404 |  |

## Appendix F Synthetic Data Generation with 2× Pool and Top- M M Selection

In this section, we provide further details regarding the number of synthetic data samples used. In generating synthetic data, we first provide five samples from the original training set as few-shot exemplars to guide the model toward producing outputs in the desired style and format. For each task, we initially create a synthetic pool containing twice the number of examples used in the source model’s training. This pool is then filtered using contrastive excess scores, after which we retain only the top $M$ samples, where $M$ equals the size of the source training set. To be specific, we set $M=250$ for BBH, $M=90$ for MMLU, and $M=200$ for LaMP tasks. This procedure ensures that the target model’s LoRA adapter is trained on a dataset comparable in scale to that of the source model, while selective filtering enhances the overall quality of the retained data.

## Appendix G ROUGE-L Filtering for Diverse Synthetic Queries

We now proceed to provide the task lists for which we did not apply ROUGE-L filtering when generating diverse synthetic queries.
In general, we use ROUGE-L filtering to encourage diversity in queries, but for the tasks listed in Table [7](https://arxiv.org/html/2510.04682v1#A7.T7), we only applied simple deduplication.
In the case of BBH, tasks such as boolean_expressions or temporal_sequences already follow highly restricted and repetitive patterns, making high ROUGE-L scores inevitable.
For MMLU, the only tasks without ROUGE-L filtering are the history-related subjects.
This is potentially because history questions often require long passages that overlap in vocabulary, phrasing, or factual references (e.g., recurring names, dates, or events).
Applying strict ROUGE-L filtering in such cases would make it difficult to generate the required number of synthetic queries.
For all remaining BBH and MMLU tasks, as well as all LaMP tasks, we applied a ROUGE-L threshold of 0.7 to encourage diversity while still preserving task fidelity.
Table [7](https://arxiv.org/html/2510.04682v1#A7.T7) provides the tasks for which ROUGE-L filtering is not applied.

**Table 7: Tasks without ROUGE-L filtering.**
| Category | Tasks |
| --- | --- |
| BBH | bbh_boolean_expressions, bbh_date_understanding, bbh_disambiguation_qa,<br>bbh_geometric_shapes, bbh_logical_deduction_three_objects, bbh_multistep_arithmetic_two,<br>bbh_navigate, bbh_object_counting, bbh_penguins_in_a_table, bbh_reasoning_about_colored_objects,<br>bbh_salient_translation_error_detection, bbh_snarks, bbh_temporal_sequences,<br>bbh_tracking_shuffled_objects_three_objects, bbh_web_of_lies |
| MMLU | high_school_world_history, high_school_us_history, high_school_european_history |

## Appendix H Prompts for Synthetic Query Generation

In this section, we present the prompts used for generating synthetic queries.
Each prompt includes five examples, which correspond to the seed prompts taken from the original data.
For BBH, which consists of multiple subtasks, we specify task-specific instructions and formatting rules so that the generated queries and labels are structured according to the expected format.
Each BBH subtask is paired with its corresponding rules and output format to ensure consistency across the benchmark, and Table [8](https://arxiv.org/html/2510.04682v1#A8.T8) illustrates the case of boolean_expressions as a representative example.

In contrast, for MMLU a single unified prompt format is sufficient for most subjects, and the model reliably generates queries that follow the expected structure. The exception is the history-related tasks, namely, high_school_us_history, high_school_world_history, and high_school_european_history.
For these tasks, it is observed that more specific prompting is required to obtain better generations.
The general MMLU prompt is shown in Table [9](https://arxiv.org/html/2510.04682v1#A8.T9), whereas the history-related tasks make use of the specialized prompt provided in Table [10](https://arxiv.org/html/2510.04682v1#A8.T10).

Finally, for LaMP tasks, we apply the same query template uniformly to each user. The detailed prompt templates for LaMP 4 and 5 are provided in Tables [11](https://arxiv.org/html/2510.04682v1#A8.T11) and [12](https://arxiv.org/html/2510.04682v1#A8.T12) respectively.

**Table 8: Synthetic query prompt for the BBH boolean_expressions task. Each BBH subtask requires task-specific instructions and formatting rules to ensure that generated queries and labels follow the expected format. We show the boolean_expressions task here as a representative example.**
| Synthetic query generation prompt for BBH (boolean_expressions) task. |
| --- |
| System |
| You are an expert task generator for boolean_expressions tasks. Generate boolean expression evaluation tasks ending with ‘ is‘ – SINGLE LINE ONLY. |
| CRITICAL FORMAT REQUIREMENTS: |
| - Follow the EXACT format structure shown in the examples. |
| - ABSOLUTELY CRITICAL: Generate ONLY ONE LINE, ONE TASK per response. |
| - NO multiple lines, NO newlines (\n), NO multiple expressions. |
| - Must end with ‘ is‘. |
| - Example of INVALID output: True or False is\n\n False and True is |
| - Example of VALID output: True or False or not False and ( True or False ) is |
| Generate diverse content but maintain the exact same format structure. Only output the task input, not the solution. |
| User |
| Generate new boolean_expressions tasks following these exact format examples:<br>Example 1: [boolean expression ending with ‘ is‘] |
| Example 2: [boolean expression ending with ‘ is‘] |
| Example 3: [boolean expression ending with ‘ is‘] |
| Example 4: [boolean expression ending with ‘ is‘] |
| Example 5: [boolean expression ending with ‘ is‘] |
| CRITICAL: Generate ONLY ONE LINE, exactly like the examples above. |
| Generate a new task following the exact format: |

**Table 9: Synthetic query prompt for the MMLU. Applied to all tasks except for history related tasks.**
| Synthetic query generation prompt for MMLU (general). |
| --- |
| Example 1: |
| Question: [question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 2: |
| Question: [question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 3: |
| Question: [question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 4: |
| Question: [question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 5: |
| Question: [question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 6: |

**Table 10: Synthetic query prompt for the MMLU history tasks only. The history tasks are high_school_us_history, high_school_world_history, and high_school_european_history. These history tasks required more specific prompting to obtain better results.**
| Synthetic query generation prompt for MMLU (history related tasks). |
| --- |
| Generate [SUBJECT] multiple choice questions following these examples: |
| Example 1: |
| Question: [history question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 2: |
| Question: [history question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 3: |
| Question: [history question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 4: |
| Question: [history question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Example 5: |
| Question: [history question snippet] |
| 1. [choice snippet] |
| 2. [choice snippet] |
| 3. [choice snippet] |
| 4. [choice snippet] |
| Answer: [1–4] |
| Now generate a new [SUBJECT] question following the same format: |
| Question: |

**Table 11: Synthetic query prompt for the News Headline Generation task.**
| Synthetic query generation prompt for News Headline Generation |
| --- |
| You are a news text generator. Generate diverse news article texts that could be used to create headlines.<br>Only output the raw news text content, not headlines or queries. |
| Example 1: [article snippet] |
| Example 2: [article snippet] |
| Example 3: [article snippet] |
| Example 4: [article snippet] |
| Example 5: [article snippet] |
| Example 6: |

**Table 12: Synthetic query prompt for the Scholarly Title Generation task.**
| Synthetic query generation prompt for Scholarly Title Generation |
| --- |
| You are a scholarly abstract generator. Generate diverse ONE paragraph abstracts that ask for creating paper titles from abstracts.<br>The abstract should be ONE paragraph only. Only output the raw abstract text, not the actual titles. |
| Example 1: Create a title for this research paper: |
| Abstract: ”[abstract snippet]” |
| Title: ”[title]” |
| Example 2: Create a title for this research paper: |
| Abstract: ”[abstract snippet]” |
| Title: ”[title]” |
| Example 3: Create a title for this research paper: |
| Abstract: ”[abstract snippet]” |
| Title: ”[title]” |
| Example 4: Create a title for this research paper: |
| Abstract: ”[abstract snippet]” |
| Title: ”[title]” |
| Example 5: Create a title for this research paper: |
| Abstract: ”[abstract snippet]” |
| Title: ”[title]” |
| Write a research paper abstract as a single paragraph containing at least 3 sentences. |
| Example 6: |

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: - How sensitive is TITOK to the quality of the source LoRA? Would a poorly tuned LoRA produce misleading excess scores?
- Could contrastive excess be used iteratively (e.g., to refine the LoRA itself) rather than just for one-shot transfer?
- How does TITOK behave when transferring between models with different architectures (e.g., dense vs. mixture-of-experts backbones)?
- Can the authors clarify the computational overhead of computing per-token log-likelihood differences compared to TransLoRA’s discriminator?
soundness: 3
strengths: - The idea of contrastive excess as a proxy for task-specific token importance is elegant and avoids the need for auxiliary discriminators or extra training modules. It’s conceptually sound and efficiently leverages existing model behavior.
- TITOK uses no extra models and operates with modest computational overhead compared to TransLoRA. This design aligns well with real-world deployment constraints.
- The authors test across multiple model families (Mistral, Llama 2/3) and tasks (reasoning, personalization), presenting clear gains in each. The inclusion of ablations and sensitivity analyses (on token selection ratio, query generation source, etc.) adds strong empirical credibility.
- The dual-pointer alignment and averaging rules are simple yet effective.
summary: This paper introduces TITOK (Transfer Token-level Knowledge), a framework for transplanting LoRA adapters across different LLM backbones by transferring token-level knowledge instead of sequence-level knowledge.
Unlike previous approaches such as TransLoRA, which require a discriminator model to filter synthetic data, TITOK leverages a contrastive excess signal to identify informative tokens that encode task-specific knowledge. TITOK requires neither extra models nor additional training overhead. It comes with an effective mechanism to resolve tokenizer mismatches between source and target models, which enhances robustness and applicability.
weaknesses: - The proposed method still relies on generating a large pool of synthetic samples, which may introduce significant costs
- While token-level filtering is the paper’s key innovation, there is no qualitative or visual analysis demonstrating which tokens are retained or why they contribute to effective transfer.

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: 1. The paper lacks a clear explanation of why the contrastive excess between the LoRA-equipped and base models effectively captures transferable task knowledge. Could the authors provide theoretical reasoning or empirical evidence (e.g., gradient or information analysis) to support its validity?

2. TITOK heavily depends on synthetic data generated by the source model. How robust is the method to variations in data quality, prompt design, or model strength? Would TITOK still perform well if the source model or synthetic data are suboptimal?

3. The algorithm’s performance appears sensitive to the top-k% token selection and sample filtering. Can the authors provide quantitative sensitivity analyses or propose adaptive mechanisms to make TITOK more stable across tasks?

4. The tokenizer alignment process is crucial but not quantitatively evaluated. How accurate is this alignment across model families with different tokenization schemes? Could small alignment errors lead to degraded performance?

5. The comparisons only include KD and TransLoRA. To better position the contribution, could the authors include or discuss results against more recent PEFT transfer methods (e.g., KD-LoRA, AdapterFusion, DoRA)? This would clarify TITOK’s true advantage.
soundness: 2
strengths: 1. The paper introduces a new perspective on LoRA knowledge transfer by focusing on token-level contrastive signals (“contrastive excess”) rather than sequence-level alignment. This provides a fine-grained and model-agnostic approach that does not require additional networks or data annotations.
2. The authors conduct systematic experiments across multiple benchmarks (BBH, MMLU, and LaMP) and various transfer settings (intra-family, cross-family, and cross-version). The results consistently demonstrate that TITOK achieves stable improvements over established baselines such as KD and TransLoRA.
summary: This paper presents a framework called TITOK for transferring LoRA-based knowledge between large language models (LLMs). The authors note that existing parameter-efficient fine-tuning (PEFT) methods such as LoRA cannot be directly transplanted across different backbones, while knowledge distillation (KD) approaches depend heavily on access to original task data. TITOK introduces the concept of contrastive excess, computed between a source model with LoRA and its corresponding base model without LoRA, to identify task-informative tokens. Using these token-level signals, the method filters synthetic data and trains the target model’s new LoRA adapter only on the most informative tokens, avoiding the need for an additional discriminator or real training data. Experiments on BBH, MMLU, and LaMP benchmarks show that TITOK consistently outperforms KD and TransLoRA across multiple transfer settings, achieving average performance gains of 4–8%. Overall, the paper proposes a lightweight token-level transfer framework and demonstrates its effectiveness through empirical evaluation.
weaknesses: 1. The paper does not provide a solid theoretical justification for why the proposed contrastive excess signal effectively captures transferable task knowledge. The formulation remains largely heuristic, and no analysis is presented to support its generalization properties or convergence behavior.

2. Although the experiments cover multiple datasets, the reported improvements (4–8%) are modest and sometimes within the margin of variance. The absence of standard deviations or statistical significance tests weakens the credibility of the claimed gains.

3. Despite claiming to reduce data dependence, TITOK still relies heavily on synthetic data generated from the source model. The paper does not adequately evaluate how data quality, diversity, or prompt design affect transfer performance, making reproducibility and robustness questionable.

4. The paper mainly compares against KD and TransLoRA but omits other recent or stronger PEFT transfer approaches. Without these comparisons, it is unclear whether TITOK’s benefits are meaningful beyond this narrow baseline set.

5. Several key implementation details—such as the choice of hyperparameters (e.g., k% token ratio) and the impact of tokenizer alignment—are only described qualitatively. The algorithmic intuition is under-explained, and the overall presentation lacks ablation depth, which limits interpretability and reproducibility.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Did the authors perform any human evaluation or external fact-checking to verify the quality of the filtered synthetic data?
soundness: 3
strengths: 1. TITOK avoids extra models or full datasets by leveraging only token-level contrastive excess scores, making it computationally efficient and easy to deploy.

2. Demonstrates consistent performance gains across diverse transfer settings, including different model families, sizes, and tokenizers, highlighting robustness and broad applicability.
summary: This paper proposes TITOK, a lightweight framework for transferring LoRA adapters across LLMs without original training data. It uses token-level contrastive excess scores from the difference between a base model and its LoRA version to filter synthetic data and focus training on informative tokens. TITOK avoids extra discriminators or full datasets, achieving consistent improvements over baselines like KD and TransLoRA across tasks and model types.
weaknesses: - Only 7–8 B models; no evidence it scales to 70 B+ or harder generative tasks.

- Source model generates both query and label; filter can’t catch wrong or biased synthetic data.

- No formal justification that “contrastive excess” equals transferable knowledge.

---

confidence: 3
contribution: 2
initial_rating: 4
presentation: 3
questions: See weaknesses
soundness: 3
strengths: 1. Lightweight and scalable.
TITOK requires no additional model training; it simply reuses the source model’s log-likelihoods. The proposed sample filtering and token selection can be implemented efficiently and applied to diverse architectures.

2. Cross-tokenizer alignment.
The tokenizer-alignment algorithm (Fig. 4) is an important practical contribution that allows reliable transfer across families like Mistral→LLaMA, addressing a frequent but rarely discussed issue in adapter transplantation.

3. Comprehensive evaluation.
The experiments cover both reasoning (BBH, MMLU) and personalization (LaMP) tasks, across intra- and inter-family transfers (same model, different family, size, and version). Ablations, sensitivity to token selection ratio, and external-data experiments are thorough and convincing.

4. Strong empirical results.
TITOK consistently improves over baselines (e.g., +7.11% average gain in cross-model transfer) and shows robustness even when transferring with unrelated data (Table 3).
summary: This paper introduces TITOK, a framework for transferring LoRA adapters between large language models through token-level contrastive knowledge transfer.
Unlike existing methods such as TransLoRA, which rely on synthetic data filtered by an external discriminator, TITOK uses a self-contained contrastive excess mechanism that compares token-level likelihoods between a source model with and without LoRA. Tokens where the LoRA contributes the most (“excessive” tokens) are identified as carrying rich, task-specific knowledge.
weaknesses: 1. Incremental relative to TransLoRA.
While the contrastive excess concept is novel, much of the overall framework (synthetic data, adapter transplantation) follows TransLoRA. The improvement may be viewed as a simplification rather than a fundamental rethinking of LoRA transfer.

2. Limited theoretical justification.
The paper could benefit from more formal analysis—e.g., why token-level contrastive loss correlates with adapter informativeness, or whether excess scores approximate information gain.

3. Evaluation scope.
All experiments use instruction-tuned LLMs up to 8B parameters. It remains unclear whether TITOK scales to larger frontier models (≥70B) or non-text modalities.

4. Minor clarity issues.
The term “contrastive excess” could be better linked to contrastive learning literature; at present it may be confusing since no contrastive pairs or negatives are explicitly optimized.


Decision: Accept (Poster)