TiTok: Transfer Token-level Knowledge via Contrastive Excess to Transplant LoRA
Authors: ChanJoo Jung
Jaehyung Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Models
Knowledge Transfer
PEFT
Primary area: foundation or frontier models, including LLMs
TL;DR: We propose a new framework TiTok, which enables effective LoRA transplantation through token-level knowledge transfer

Abstract: Large Language Models (LLMs) are widely applied in real world scenarios, yet fine-tuning them comes with significant computational and storage costs. Parameter-Efficient Fine-Tuning (PEFT) methods such as LoRA mitigate these costs; however, the adapted parameters are dependent on the base model and cannot be transferred across different backbones. One way to address this issue is through knowledge distillation, but its effectiveness inherently depends on training data. Recent work such as TransLoRA avoids this by generating synthetic data; nevertheless, this adds complexity since it requires training an additional discriminator model. In this paper, we propose TiTok, a new framework that enables effective LoRA Transplantation through Token-level knowledge transfer. Specifically, TiTok captures task-relevant information through a token-wise contrastive excess between a source model with and without LoRA. This excess highlights informative tokens and enables selective filtering of synthetic data, all without additional models or overhead. Through experiments on three benchmarks across multiple transfer settings, we demonstrate that TiTok is consistently effective, achieving average performance gains of +4–10% compared to baselines overall.

Published as a conference paper at ICLR 2026
TITOK: TRANSFER TOKEN-LEVEL KNOWLEDGE VIA CONTRASTIVE EXCESS TO TRANSPLANT LORA
Chanjoo Jung', Jachyung Kim!
Yonsei University
{chanjoo0422 427,, jaetyunggJeyonseei.ce
ABSTRACT
Large Language Models (LLMs) are widely applied in real world scenarios, yet fine-tuning them comes with significant computational and storage costs. Parameten-Efffiieet Fine-Tuning (PEFT) methods such as LoRA mitigate these costs; however, the adapted parameters are dependent on the base model and can- not be transferred across different backbones. One way to address this issue is through knowledge distillation, but its effectiveness inherently depends on training data. Recent work such as TransLoRA avoids this by generating synthetic data; nevertheless, this adds complexity since it requires training an additional discrim- inator model. In this paper, we propose TITOK, new framework that enables effective LoRA Transplantation through Token-level knowledge transfer. Specif- ically, TITOK captures task-relevant information through token-wise contrastive excess between a source model with and without LoRA. This excess highlights informative tokens and enables selective filtering of synthetic data, all without ad- ditional models or overhead. Through experiments on three benchmarks across multiple transfer settings, we demonstrate that TITOK is consistently effective, achieving average performance gains of + 4-10% compared to baselines overall."
INTRODUCTION
Large Language Models (LLMs) (Brown et al., 2020; Vaswani et al., 2023) have made significant progress in many real-world applications, including chatbots (OpenAl al., 2024), search engines (Xiong et al., 2024), and coding assistants (Rozière et al., 2024). While fine-tuning LLMs has been demonstrated to be a promising way to improve performance on downstream tasks, it incurs sub- stantial computational and storage costs. To alleviate this, Parameter- (Houlsby et al., 2019) methods such as LoRA (Hu et al., 2021) update only a small subset of param- eters while keeping the base model frozen. However, PEFT's adapted parameters are dependent to the base model and thus cannot be transferred across different models. This limitation is increasingly critical given the rapid release of new L.MMs and the growing diversity of available models.
Fine-Tuning (PEFT)
One potential approach to mitigate this limitation Knowledge distillation (KD) (Hinton et al., 2015; Azimi al., 2024), which transfers the knowledge embedded in source model's PEFT adapters to a target model with new PEFT adapters by aligning the target's output distributions with those of the source. However, KD is inherently data-dependent, typically requiring access to training data from target downstream tasks (Nayak et al., 2019; Liu et al., 2024). which is often unavailable or costly obtain. To address this limitation, Tra (Wang al., 2024) has recently proposed to use synthetic data, leveraging the data synthesis capabilities of recent L.LMs (Wang et al., 2023; Kim al., 2025). This approach enables the target model to acquire domain knowledge without direct access to the original dataset. Nevertheless, TransLoRA requires training an additional discriminator model to filter low-quality synthetic data, which inevitably introduces extra complexity and computational overhead. Furthermore, it primarily emphasizes the role of synthetic data, paying less attention to how the knowledge transfer process itself should be designed.
Contribution. In this paper, propose Transplantation through Token-level knowledge transfer (TIToK). Our high-level idea is to se-
framework that enables effective LoRA
Code available https: //tpt/gttttt www.ub..coooaaao wtt.ub...oo//agghttyaatttiiii. 6/TiTok

---

Published as a conference paper at ICLR 2026
GOAL
5 seed prompts
IF
source_ bese
LoRA
LoRA Knowledge Transfer
source bass
Contrastive Excess Filtering Mechanism
Tokenization Alignnent
Local
Man Buya Treadmill
target base LoRA
Synthetic Data Generation
Sample Filtering
Token Selection
Target Model Learning
Figure I: Overview of TITOK: Transplantation through Token-level knowledge transfer. Starting from small set of seed prompts, the source expert model (source base model LoRA) generates synthetic data. A token-wise contrastive excess filtering mechanism then compares the expert against its base backbone compute token-level excess scores. Using these scores, TITOK first per- forms sample filtering and subsequently token selection, retaining only the most informative samples and tokens. When tokenizers differ, masks are aligned prior training. The resulting filtered data is finally used to train a new LoRA on the target backbone, enabling efficient knowledge transfer.
lectively convey task-relevant information from the source model's LoRA by using token-level sig- nals to guide the transfer process, rather than relying on the entire token sequence. We specifically capture this information through a concept we introduce as token-wise contrastive excess, which is obtained by comparing predictions from a source model with LoRA and the same model without LoRA. Intuitively, this token-win wise contrastive excess highlights tokens that contain important task knowledge. This excess signal further utilized to filter the generated synthetic data for training, thereby enabling selective learning samples that contain richer information. Unlike TransLoRA, which requires training an additional discriminator model for filtering, TrToK requires neither extra models nor additional training overhead. Moreover, design effective mechanism to resolve tokenizer mismatches between source and target models, enhancing robustness and applicability.
We demonstrate the effectiveness of TITOK by conducting extensive experiments three widely used benchmarks, which cover both reasoning (Big Bench Hard (Suzgun et al., 2022) and MMLU (Hendrycks et al., 2021)) and personalization tasks (LaMP (Salemi al., 2024)). In particular, TITOK improves the performance by +9.94% over the vanilla target model, +8.5% over KD, and +4.4% over TransLoRA, when averaged across all tasks and transfer settings. We also explored variety of transfer settings, including transfers within the same model, across different model families and sizes, and even across different model versions. In every case, our approach achieves consistent improvements. Interestingly, TITOK remains effective even when applied to external data originating from tasks different from the target task, highlighting both robustness and general applicability. Overall, these empirical results highlight TiToK as methodologically simple yet powerful paradigm for efficiently transferring LoRA knowledge across models in diverse scenarios.
2 RELATED WORKS
Transferring PEFT adapters. Parameter-Efficien Fine-Tuning (PEFT) (Houlsby et al., 2019; Li & Liang, 2021) has emerged as both practical and popular alternative to full model fine-tuning. By requiring updates to only small portion of parameters, it enables efficient adaptation, with LoRA (Low-Rank Adaptation) (Hu et al., 2021; Dettmers et al., 2023) standing out as one of the most widely adopted methods. However, fundamental limitation is that LoRA adapters to the frozen backbone they were trained on, making them difficult to transfer to other base models.
tied
To address this issue, recent studies such ToA (Wang al., 2024) have attempted to
2

---

Published as a conference paper at ICLR 2026
transplant the knowledge of LoRA adapters across models by generating synthetic data (Wang et al., 2023). While effective to some extent, this approach requires an additional discriminator model to filter high-quality synthetic data, resulting in relatively heavy pipeline. In parallel, Knowledge Distillation (KD) (Hinton et al., 2015; Azimi et al., 2024) has been widely explored as another means of knowledge transfer; however, traditional KD typically operates within teacher-student framework at the logit or sequence level and requires access to training data close to the original, in order use the teacher's distribution for supervising the student. In contrast, our approach focuses on token-level selective transfer, offering more fine-gr theeggrined yet lightweight alternative that enables efficient transplantation of LoRA adapters across diverse models for deployment.
Data synthesis with LLMs. Synthetic data generation with LLMs has attracted increasing atten- tion as a means reduce reliance on costly or inaccessible datasets (Wang al., 2023; Kim al., 2025). Prior lines of research have leveraged synthetic data for purposes such as privacy preser- vation (Bu et al., 2025), data augmentation (Kumar al., 2021), and domain adaptation (Li al., 2023). In our framework, synthetic data serves as a core component, enabling the transfer of LoRA adapters without access to the original training corpus, while simultaneously mitigating privacy con- cerns and reducing the degree of dependency labels are generated directly by the source expert model itself. synthetic data ultimately provides a self-sufficient mechanism that fits to our objective of lightweight and effective knowledge transfer.
external datasets. Moreover, since both queries and
Selective token training. Recent studies (Lin et al., 2025; Gu et al., 2020) demonstrate that not all tokens contribute equally to model training, motivating research on selective training strategies. While such approaches have primarily been applied to accelerate optimization or reduce redundancy (Yeongbin et al., 2024; Bal et al., 2025), work innovatively extends this concept to the setting of knowledge transfer. Specifically, our concept of excess scores (Eq. 2) is derived from this idea, where the contrast between the source backbone and its LoRA adapter yields token-level judgments. This enables TrTok to transplant LoRA knowledge in a more focused and fine-grained manner, highlighting the broader applicability of selective training beyond its original scope.
3 TITOK: TRANSPLANTING LORA THROUGH TOKEN-LEVEL KNOWLEDGE
In this section, we introduce TYTOK, a framework for LoRA Transplantation through Token-level knowledge transfer (Fig. 1). The core idea of TITOK is to transfer the knowledge from a source model's LoRA adapter into a target model's LoRA adapter by training selectively on the informative tokens within synthetic data. Specifically, the framework consists of three components: (I) Syuthetic data generation (Sec. 3.1), where the source expert model produces query-label pairs for target task; (2) Excess score computation (Sec. 3.2), which calculates token-level importance using the source model; and (3) Target model training with filtering (Sec. 3.3), which trains the target model with newly initialized LoRA adapter on top-ranked samples and tokens. addition, we propose Excess score alignment (Sec. 3.4), algortihm designed to apply TITOK even when tokenizers of the source and target models differ. The overall algorithm of TITOK is presented in Alg. I.
3.1 SYNTHETIC DATA GENERATION VIA LLM PROMPTING WITH FEW-SHOT DATA
Let M, denote the source backbone LLM and A, its LoRA adapter target downstream task, which forms the source expert model M, + Ag. The target model, whose LoRA adapter A, will be trained, is denoted by M₁. Then, TITOK first constructs synthetic dataset D,, similar to the idea of Toa ( (Wang et al., 2024). This usage of synthetic data allows us to avoid keeping the whole original dataset for the downstream task, and simultaneously let A learn the knowledge encoded in the source adapter A,. Unlike ToansLoRA's approach of using the untuned target model Mr to generate synthetic data, we use the source expert model MM + A. to synthesize data (see empirical comparison in Fig. 3). Concretely, M2 + A, synthesizes both the query and the label within a prompting- based data synthesis framework (Wang et al., 2023) (see details in Appendix P): given a few-shot data of downstream task, it first generates a query q. and then produces the corresponding label y conditioned on q. To encourage diversity, we apply ROUGE-I. filtering together with dedu- plication to all tasks, except for exceptional cases where such filtering is infeasible (more details are in Appendix I). Consequently, the resulting synthetic dataset consists of query-label pairs:
D. =
3

---

Published as a conference paper at ICLR 2026
In this way, Mr would be trained on the synthetic data D, generated by the source expert model M, + Ag, thereby enabling knowledge transfer without relying on the entire original dataset.
3.2 TOKEN-WISE CONTRASTIVE EXCESS SCORE FROM SOURCE MODEL WITH LORA
As described in Sec. 3.1, TiToK relies synthetic data, but synthetic data is often prone to imperfections (Chen et al., 2024b); thus, sophisticated filtering is essential to retain high-quality informative samples. A (Wang et al., 2024) tackles this challenge with a separate discrimi- nator to filter useful queries, but this introduces the extra burden of training and maintaining an extra model. In contrast, we propose a light alternative that only utilizes the already trained source model. Specifically, we use source model and its LoRA adapter to perform two complementary roles: (1) the amateur role (M,) and (2) the expert role (M, + A.). Then, the difference between the two roles provides an implicit supervision signal where the task information is encoded.
Formally, yet = [,)....,,91 yr.] denote the synthesized response, where yi is a token of corresponding to the synthesized query q. Then, we define the excess score as:
S(y,) (v. Ly(y))
(2)
where the amateur and expert losses token y; are defined as:
La(y)) log (M q. i), Le(ys) log PM. q,y< i).
(3)
The excess score S(yi) quantifies the knowledge discrepancy incurred by equipment of LoRA, therehy identifying the tokens where the adapter provides a decisive contribution. Intuitively, if the backbone model uncertain about predicting a token but the LoRA-enhanced model assigns with high confidence, that token will thus obtain large excess score. This implies that tokens with higher S(ys) correspond to positions where the LoRA adapter injects task-specific knowledge that the backbone could not capture on its attribution signal, derived entirely from the internal behavior the model itself, and guides training toward the specific regions of data that are most enriched with the adapter's knowledge.
In this way, the excess score functions a fire-grained
While the excess score motivated from an intuitive contrast between the backhone and its LoRA-enhanced counterpart, it is also grounded in principled statistical theory. Theoretically, the proposed token-wise contrastive excess score is connected to a token-level log-likelihood ratio (LLR) between the source expert (backbone LoRA) and the backbone. LLR robust metric widely established in statistical testing (Li & Babu, 2019). For example, by the Pearson lemma (Neyman & Pearson, 1933), the likelihood ratio is known identifying differences between two models. Therefore, high-LLR tokens are exactly the regions where the adapter changes the predictive distribution, i.e., where task knowledge injected. In addition, from the perspective of standard knowledge distillation, tokens with nearly zero L.RR provide no additional teacher knowledge and offer no benefit for transfer. In contrast, high-LLR tokens correspond to maximal teacher-student divergence, larger gradients, and higher information contribution to the student. Therefore, emphasizing tokens with high LLR in particular naturally focuses training the precise positions where transferable knowledge is concentrated.
the optimal statistic for
3.3 TARGET MODEL TRAINING WITH SAMPLE FILTERING AND TOKEN SELECTION
After the computation of excess scores S(y:), the newly initialized LoRA adapter A, for target model M, is trained using the synthetic samples (q,. y,) Dr with two-level filtering schemes.
First stage: Sample filtering. We begin by filtering the synthetic dataset D. (Eq. 1) at the sample level remove less informative examples. For each synthetic sample, we compute the mean of the excess scores S(y) across the tokens in and retain only M samples with the highest values.
S
Σ S(y)).
(4)
BiEY;
Let Dr be the set of the M samples in D. with the largest Syz
Dj = M (q/.y)) E Da }.
(5)
Through this step, the synthetic data undergoes a filtering process, ensuring that subsequent training is concentrated specifically the remaining examples in D, with richer knowledge signals.
4

---

Published as a conference paper at ICLR 2026
Algorithm 1: TTToK: Transplanting LoRA through Token-Level Knowledge
Input: source expert M, + A., larget Mr. parameters N, M, 6%
Output: trained target LoRA Ar
Construct synthetic dataset D. ([q));))}3 with M. + A.
for (q,,,, D. do Compute token excess scores (3y = Le (3c) - (3c)
Calculate mean score S, 1,, E,, Ry; S(v.)
3. Select lop-M samples by S, to form D;
4. for (q,,,, ) 2D do
Rank tokens by S(3x) and keep top-k%, represented by mask I0% (26)
(okenizer(MM)) # tokenizer(MM then
Align masks 1/% -
[ (1:)
6. Train At on Mr with masked loss LrsTok
Σ (.
Σ
(u:((:)) Le(ss)
return Ar
Second stage: Token selection. Next, we consider token selection; that is, Ar does not learn from all tokens within the retained samples. Instead, it focuses only on those prioritized by the excess scores S(y),, which identified most important for knowledge transfer. To achieve this, select the top k% of tokens ranked by their excess scores using the indicator (3:):
(1, if ranky, ffrnkky(S(t)) (S(y:) S 1&% ly,ll,
15% (W:)
(o.
(6)
otherwise,
where Iy, denotes the number of tokens in y, and ranky, (S(y:) indicates the rank of S(y)) among the tokens of that response. Based on this selection, the training objective for Ar is defined as
CTITOK= Σ Σ (xx(y)) Le(3)),
(E MEE,
where Le(y)) is the negative ligeliild loss assigned by M, A, on token Yi (only A, is learnable). By training only on these filtered tokens (Eq. 7). TTToK enables A, to efficiently acquire the source LoRA's knowledge without access to the original training data or any external models.
3.4 EXCESS SCORE ALIGNMENT ACROSS DIFFERENT TOKENIZERS
In cases of transfer between models with different tokenizers, direct mapping of token-level signals not possible, given that the source and target models may segment texts differently. To address this, introduce simple yet robust tokenizer alignment algorithm that propagates token masks (Eq. 6) from the source token sequence (()) to the target token sequence (f)), The algorithm first aligns token sequences using dual pointers that incrementally decode and match text spans. Masks then propagated using the following four rules: (1) direct copy for one-to-one mappings, (2) replication for man-to (3) averaging for many-to-one, and (4) averaging with replication for many-to-many Finally, top-k% selection step retains the most confident target tokens. This process ensures consistent supervision across tokenizers, enabling reliable transfer even when models tokenize text differently. The conceptual illustration of this procedure is presented in Fig.4.
EXPERIMENT
In this section, we present our experimental results to answer the following research questions:
RQI: Can TITOK efficiently transfer knowledge of LoRA in various scenarios? (Table 1)
RQ2: What is the contribution of each component in TrToK? (Table
RQ3: How sensitive is token-level selective transfer to the selection ratio? (Figure 2)
RQ4: How does the choice of model to synthesize query affect the performance? (Figure 3)
RQ5: Can TITOK transfer knowledge using data from a different unrelated domain? (Table 4)
5

---

Published as a conference paper at ICLR 2026
4.1 EXPERIMENTAL SETUPS
Models. We mainly present our knowledge transfer experiments using models from the Mistral and Llama families. Specifically, we designed the following LoRA transfer (source - target) setups. + distral-7B-Inst-vol + L-ama-311-8--nnst: the different-family model transfer setup, (3) Llama-3.2-3B-Inss
(1)
the basic transfer setup, (2) Mistral-7B-Inss In
+ + Llama- 3.I-:B-Inst: the different-versiae model transfer setup. These setups are intended to test whether a smaller model can effectively transfer knowledge to larger one, and to explore whether a relatively weaker model can still influence a newer, stronger model. These various setups realistically reflect how LLMs develop today, where various models released and newer, improved versions keep emerging. Model names are abbreviated for clarity and are used consistently in all tables and 2) "Ilama3 8B" Llama-3. 1.1-a---.1----..... 1-8B-Instruct, 3) "I.Jama3 3B" = 2a-ma-3.2-31--Intttte 4) "[lama2 7B = 1lama--77-ccht-hh
Llama-3.1-8B- the different-size model transfer setup, and (4) Llama-2-7b-chatt
figures: 1) "Mistral 7B" Biistral-B--Instrutt-ea
Baselines. To demonstrate the effectiveness of TITOK, we compare it against three baselines: (i) Vanilla, the target base model without any fine-tuning; (ii) KD(+MinED), knowledge distillation (KD) from source expert model (Hinton et al., 2015). When the source and target models use dif- ferent tokenizers, the original KD is not applicable. In this case, use Minimum- -ddt---itttttteeeeee (MinED) tokenizer alignment (Wan et al., 2024), which aligns both token sequences and vocabulary distributions via aanamii-programmeet and by mapping probability mass to nearest edit-distance neighbors (e.g.. "immediately" ++ "imme- diate"). We use the synthesized data by TransLoRA as the training data for KD (+MinEE) ; and (iii) TransLoRA (Wang et al., 2024), a prior method in which the vanilla target model synthesizes the queries, while the source model with its LoRA adapter generates the corresponding synthetic labels. A discriminator is then employed to filter the synthetic data for training the target LoRA adapter.
sequence alignment for near matches (e.g., "gets" ++ "get")
Datasets. Following prior work in TransLoRA, we first conduct experiments on two represen- tative benchmarks: (1) Big-Bench Hard (BBH) (Suzgun et al., 2022) and (2) Massive Multitask Language Underslunding (MMLU) (Hendrycks et al., 2021). BBH consists of 27 challenging rea- soning tasks structured as multiple choice or short answer questions, designed to test compositional generalization and advanced problem solving abilities of a Language Model. Mcanwhile, MMLU covers 57 tasks across diverse academic subjects, presented in multiple choice format to evaluate broad knowledge and reasoning skills of model. As both benchmarks provide only test sets, we construct 90%/10% train-evaluation splits. For BBH, we use the full dataset, whereas for MMLU, randomly sample 100 instances of the dataset for each task before applying the 90%/10% split.
To extend our approach to personalization and text generation, we additionally conduct experiments on LaMP benchmark (Salemi et al., 2024), focusing exclusively its generation tasks. In particular, we experiment with (3) News Headline Generation (LaMP 4) and (4) Scholarly Title Generation (LaMP 5), as they are the only text generation tasks that are both accessible and reliably evaluable. The remaining LaMP tasks are excluded, given that LaMP 1,2,3 are discriminative, and LaMP 6.7 lack gold labels. For LaMP tasks, the source expert model M, + A,, trained on data from the 30 users with the longest activity histories. From each user, we use 200 data points for training and 50 data points for validation, design choice intended to conduct a more rigorous and robust evaluation. In total, each LaMP task contains 6,000 training examples and 1,500 evaluation examples.
To assess performance on the BBH and MMLU, we measure the average accuracy using the LM-Eval Harness (Gao et al., 2024). Following the setup used in TransLoRA (Wang et al., 2024), all tasks are conducted in a zero-shot setting. Mcanwhile, for the LaMP tasks, we adopt ROUGE-1 and ROUGE-L scores as evaluation metrics, following the benchmark's 's primary evaluation metrics.
Implementation details. When training both the source and target models, we use a learning rate of 5 x 10-5, train for 2 epochs with a batch size of 4, and apply LoRA with rank r = 8, scaling factor a 8, and dropout 0.05. Optimization is performed using AdamW with weight decay
hntpps: hetttss://ugggingaace..o http://hhggggaaooooaaaaaaaaaaaaa
Instrruct http:///ha...ccc..aoaaaaaaaaaa// Instruct
hhtrps::/hhgginngffcc..o wwco/metaallae
"https://huggingffa httt::////ooggnnnaeeeecccottteen

---

Published as a conference paper at ICLR 2026
Table 1: Main results. Experiments on BBH, MMLU, News Headline and Scholarly Title Genera- tion tasks under four transfer settings, Reasoning tasks (BBH, MMLU) are evaluated via LM-Eval Hamesss, and personalization tasks (News Headline, Scholarly Title Generation) via ROUGE- 1/L. sero-shot evaluations are reported as the mean across 3 random seeds. Best scores are in bold.
BBH
MMLU
News Headline
Scholarly Title
Transfer
Method
Acc.
Acc.
ROUGE-1
ROUGE-L
ROUGE-1
ROUGE-L
Vanilla
0.397
0,557
0.117
0.101
0.381
0,311
Mistral 7B
KD (+MinED)
0,417
0,560
0.117
0.104
0.385
0.310
Mistral 7B
TransLoRA
0.416
0.534
0.156
0.137
0.447
0.382
TITOK (ours)
0.424
0.561
0.161
0.143
0.473
0.413
Vanilla
0,469
0,469
0.125
0.110
0.444
0.378
Mistral 7B
KD (+MinED)
0.475
0,482
0.127
0.112
0.454
0.387
Llama3 8B
LoA
0.473
0.473
0.126
0.110
0.461
0.397
TIToK (ours)
0.484
0.485
0.139
0.123
0.464
0,403
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Llama3 3B
KD (+MinED)
0.474
0.477
0.125
0.110
0.449
0,383
Llama3 8B
TransLoRA
0.471
0,467
0.122
0.108
0.454
0.387
TiTOK (ours)
0.496
0.478
0.127
0.113
0.456
0.392
Vanilla
0.469
0,469
0.125
0.110
0.444
0.378
Ilama2 7B
KD (+MinEDD)
0.473
0.476
0.125
0.110
0.449
0.382
Llama3 8B
Trans A
0.472
0.468
0.123
0,109
0.457
0.394
TITOK (ours)
0,488
0.477
0.138
0.120
0.461
0.403
1 10-2, together with a linear learning rate schedule and warmup ratio of 0.1. For synthetic data generation, we provide five samples from the original training data as few-shot exemplars, and apply top-p sampling (Holtzman al., 2020) generate both queries and labels, with sampling hyperparameters tuned individually for each task. To further encourage diversity, we apply ROUGE- filtering with a threshold of 0.7 and deduplication to remove redundant queries, following Wang et al. (2023). For tasks where ROUGE-based filtering is infeasible, we apply only deduplication (see Appendix I). For the initial synthetic pool, we generate 2M synthetic samples and, after filtering with token-wise contrastive excess scores, retain the top M, where M equals the source training set size (see Appendix H). For token selection, the selection ratio k% is fixed at 70% across all tasks and transfer settings, except for the Llama3 3B + Llama3 8B transfer setting, where find that 1% = 30% consistently yields the best performance. When inferencing during the evaluation stage, adopt greedy decoding ensure fully deterministic and reproducible inference results.
4.2 MAIN RESULTS
Table summarizes the experimental results BBH, MMLU, and LaMP (News Headline and Scholarly Title generation) tasks across four transfer settings. First, we observe that transfer within the same model family is highly effective; when transplanting the LoRA adapter trained the Mistral 7B into a fresh instance of the same model, TrTOK consistently surpasses all baselines. In particular, TITOK improves the vanilla model by 23.94% average across all tasks, and further outperforms the KD and ToAA baselines by 21.95% and 4.75%, respectively. Taken together, the findings show that, first step, transfer within the same family is reliably successful.
Beyond same family transfer, we also find that TITOK is highly effective in cross-model transfer settings. For example, when transferring from Mistral 7B to Llama 8B, TIToK delivers an average improvement across all tasks of 6.79% over the vanilla model, while outperforming KD by 4.69% and LRA by 4.86%. This highlights that TIToK is not confined to intra-family knowledge transfer, but can successfully bridge across architectures. In the case of Llama3 3B to Llama3 8B, TITOK yields average gains 3.07% against vanilla, 2. 18% against KD, and 3.02% against ToA.sLoRA. These results suggest that TITOK scales effectively with model size, making it useful when moving from lightweight to larger models without losing efficiency. Finally, when transferring from Llama2 7B to Llama3 8B, TiTok performs average advantages of 5.95% over vanilla, 5.17% over KD, and 5.13% over TrAnsLoRA. This indicates that TITOK adapts robustly even across differ- model versions, implying practical relevance when models are upgraded in real-world pipelines.

---

Published as a conference paper at ICLR 2026
Table 2: Ablation study. "Sample filtering" uses the mean token-wise contrastive excess score to remove uninformative examples, while "Token selection" further refines the retained data by selectively keeping only the most informative tokens within each sample. Without sample filtering, data are randomly sampled. Results are reported accuracy (Acc.) for BBH and MMLU, averaged across tasks. Meanwhile, we report ROUGE-1 (R-1) and ROUGE-L (R-L) for LaMP tasks.
Settings
BBH
MMLU
News Headline
Scholarly Title
Sample filtering
Token selection
Acc.
Acc.
R-1
R-L.
R-1
R-I.
x
0.458
0.485
0.133
0.117
0,456
0.393
V
0,463
0.496
0.137
0.121
0.460
0.397
X
0.470
0.500
0.139
0.122
0.460
0.397
0.483
0,501
0.142
0.125
0.464
0.403
Table 3: Excess-score token subgroup analysis. Effect of token subgroup selection in Mistral 7B + Llama3 8B transfer. The top 20% excess-score tokens yield the best performance BBH.
Transfer
Method
Subgroup
Acc.
Vanilla
0.469
top 20 %
0.482
Mistral 7B - Llama3
TiToK
bottom 20%
0.468
random 20%
0.476
Collectively, these results provide strong evidence that TIToK not only excels in transfers within the same model family, but also demonstrates broad effectiveness in cross-model transfers. thereby highlighting its robustness across wide spectrum of model scales, versions, and families.
4.3 ADDITIONAL ANALYSES
In the following sections, present additional analyses to showcase the effectiveness of TiToK. Unlike the main table which reports the average of three random seeds to ensure statistical stability, the empirical results discussed in the subsequent sections are reported using single, fixed seed.
Ablation study. To validate the contribution of each component in our framework, we conduct additional experiments by selectively excluding the sample filtering and token selection mechanisms in Sec. 3.3. We report the average performance scores across all four transfer settings, and the results are presented Table 2. When sample filtering is not applied (Ist and 2nd rows), the training data for the target model is randomly sampled from the full synthetic dataset. Under this setting, applying only token selection (2nd row) performs noticeably better than the purely random baseline. This demonstrates the effectiveness of our token selection approach and empirically confirms that token- wise contrastive excess reliably identifies and selects the most informative tokens. Similarly, when only sample filtering is applied (Ist and 3rd rows), the results improve over pure random sampling. indicating that selecting high-quality data is essential. This finding further implies that incorporat- ing an effective filtering mechanism for synthetic data is essential. In our framework, token-wise contrastive excess serves this role by reliably selecting samples with richer and more informative signals, demonstrated clearly by the empirical results. Finally, the results show that combining both stages (4th row) achieves the best performance, confirming their complementary roles in filtering high-quality examples and selecting informative tokens for effective knowledge transfer.
Next, we also evaluate whether high excess-score tokens indeed carry concentrated transferable knowledge. We verify this empirically by contrasting token subgroups of the top 20%, bottom 20%, and random 20% on BBH in the Mistral 7B + Llama3 8B transfer setting, To prevent unintentional overlap across groups and to extract the most informative region, we employ a comparatively low 20% threshold for this analysis. The results presented in Table 3. Analytically, the empirical results show that the top 20% produces the best transfer performance, validating tat tokens with high token-wise contrastive excess scores indeed contain concentrated task knowledge.
8

---

Published as a conference paper at ICLR 2026
0.122
0.44
0.52
Aug. Accuracy
Avg. Accuracy
0.43
ROUGE-L
0.118
0.49
0.114
0.43
0.40
0.47
0.10
10
20
50
50
80
80
00
100
Token Selection Ratio k(36]
Token Selection Ratio k(%)
Taken Selection k(%)
(a) BBH (accuracy) Mistral 7B + Mistral 7B
(b) BBH (aceurncy)
News Headline (ROUGE-L) Llama2 7B + Llamaš 8B
Llama2 7B - Llamas 8B
Figure 2: Representative performance trends across %%. Among the three graphs, two come from the same task (BBH), while two share the same transfer setting (Llama2 7B - Llama3 8B).
Interestingly, we observe that truly important tokens are consistently selected regardless of the expert's model. For example, for BBH datasets, Mistral 7B and Ilama2 7B experts agree on 59.76% of the chosen tokens even when we limit the selection to a small top 20% subset to prevent trivial overlap. This implies that token will continue to be recognized across models if it is truly significant for the task, and it can be effectively identified by our token-wise contrastive excess score.
Effect of token selection ratio. We now proceed explore the impact of token selection ratios (k%) in our framework. Fig. presents three representative curves: one task (BBH) under two transfer settings, and another task (News Headline Generation) under the same transfer setting for comparison. For BBH with the same backbone on source and target (Mistral 7B-Mistral 7B), moderate token selection ratio of 30-70% yields the best results. Very low ratios (10-20%) lead to underfitting, while 100% ratio is ineffective as it applies no filtering at all. Interestingly, in weak-to-strooee to-strong transfer (Llama2 7B Llama3 8B) for the same BBH task, performance improves as k% decreases. This suggests that selecting only the tokens with the highest excess loss effectively filters out the majority noisy regions where weaker model is uncertain. By discarding the majority of tokens where the weaker model lacks confidence, TITOK significantly reduce neg- ative transfer. This trend, however, reverses for the News Hcadline Generation task under the same weak-to-sture stront transfer setting. For this task, larger k% is generally better, potentially because even a weaker model can provide valuable lexical and stylistic cues that are useful for personal- ization. Consequently, unlike reasoning-focusee tasks such BBH, which sensitive to noisy supervision, personalization tasks like News Headline Generation benefit more broadly from the source model's outputs, even when the source model is relatively weaker than the target model.
Impact of query generation model. We now ex- amine how the choice of query generation model in- fluences performance. Practically, synthetic queries can be generated either by the source model or by the target model, with the latter being the approach originally adopted ToansLoRA's pipeline (Wang et al., 2024). Figure 3 compares these two op- tions across different transfer settings, with the re- ported scores averaged over all BBH tasks. No- tably, we observe that using the source expert model (source backbone LoRA) for query generation generally yields stronger performance than using the target model. One possible explanation is that when synthetic data is generated by the same model, they remain closer to training distribution. This alignment between queries and labels likely pro- vides more coherent supervision, thereby facilitat- ing more effective transfer. Ultimately, these findings empirically justify our design choice to use source model as both the query and label generator, instead of using the target model.
0.54
Targer query
0.52
Avg. Accuracy
0.50
0.18
0.46
0.44
0.42
0.40
Matrel 7B
Mistrel 70 -aama3 BB
Llamed db -Uama3 BB
Uame2 Uamaa3 BB
Mistral 73
Figure 3: Impact of query souree. Across transfer settings, using the source expert model to synthesize query generally yields better performance. The accuracy averaged over all BBH tasks are reported.

---

Published as a conference paper at ICLR 2026
Effectiveness of transfer through exter- nal data source. We further examine whether TIToK can transfer knowledge effectively in cases where synthetic data is not preferred, and thus external data is used as an alternative. To this end, evaluate three alternative settings on LaMP tasks for Mistral 7B-Mistral 7B transfer setup: (1) using data from randomly chosen different user, (2) mixing data from multiple users, and (3) transferring across tasks, where data of Scholarly Title Generation is used to train on News Headline Generation and vice versa (i.e., out-of-distribution scenario). The results presented in Table 4. Remarkably, TITOK consistently outperforms all baselines across these
Table 4: Transfer using external data. ROUGE-1 (R- 1) and ROUGE-L (R-L) News Headline and Schol- arly Title Generation under three data settings for Mis- tral 7B Mistrall 7B: (1) other-user, (2) mixed-user, (3) cross-task. Best scores are highlighted bold.
News Headline
Scholarly Title
Data Setting
Method
R-1
R-L
R-1
R-L
Vanilla
0.117
0.101
0.381
0.311
Other-user
KD
0.127
0.113
0.405
0,333
TiTok (ours)
0.151
0.136
0.481
0.425
Vanilla
0.117
0.101
0.381
0.311
Mixed-user
MinED
0.124
0.110
0.409
0.337
TYTOK (ours)
0.151
0.137
0.480
0.422
Vanilla
0.117
0.101
0.381
0.311
Cross-task
MinED
0.118
0.106
0.403
0.331
TITOK (ours)
0.133
0.120
0.450
0.383
heterogeneous external settings. These findings demonstrate that TiToK is not restricted to syn- thetic data scenarios, but can also adapt effectively under external or user-provided data conditions. This highlights the fiexibility of TITOK for practical deployment in diverse real-world application scenarios and further underscores even when external data is used as an alternative to synthetic data.
adaptability across different data conditions, as is effective
Transfer under cross-architecture and large- scale. We additionally conduct experiments on larger-scale models and architectures to reinforce TITOK's effectiveness. Due to the large model size, all experiments in this section are evaluated on BBH using 4-bit quantization. As shown in Table 5. TITOK successfully transfers token-level knowledge in both settings. Specifically, cross- architecture transfer from dense model (Mistral 7B) to a mixture-of-experts (MoE) model yields a 1.98% improvement over the target baseline, while scaling up to the 70B transfer setting, achieves 4.72% gain. These results confirm that TITOK generalizes robustly across both complex MoE architectures and massive model scales.
Table 5: Larger scale transfer settings. BBH evaluation results for architcccieeeeere and large-scale model transfers.
Transfer Setting
Method
Acc.
Mistral 7B
Vanilla
0.454
Mixtral 8x73
TITOK (k=70%)
0.463
1lama-3.3-702
Vanilla
0.593
1..mma-333-000 TITOK (k=70%)
0.621
5 CONCLUSION
In this paper, we propose TITOK, a novel and efficient framework that transfers LoRA knowledge from a source model to a target model by training only on selectively chosen set of highly informative tokens. At its core, TTTOK strategically leverages token-level signals to distill task-relevant information from the source adapter, rather than indiscriminately relying on the entire token sequence. With this simple yet effective design, TTToK consistently surpasses baselines in various transfer settings, providing practical solution for efficient knowledge transfer.
Limitations and Future Work While TYTOK has shown clear advantages, opportunities remain refine the framework. First, generating synthetic data still requires few seed examples. Never- theless, this reliance is modest, as we avoid using full datasets. In addition, our experiments show external data serves as an effective alternative, reinforcing the flexibility of TITOK. Regarding to- ken selection, while TITOK currently relies on simple yet highly effective fixed threshold for token selection, future work could explore adaptive thresholding strategies to further enhance efficiency.
/nttps::/hugginngfacc... co/mi stralalaaa..............aa aalaalalaa http//ixxralllaaa Inst:ttttrttuutoeeoeooe ruct-v0. 'https://hugging ww/.co/m/////a// amaa 1.1am--3 1. 3..3- 3-70-- Instruct.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
We conduct this research in full accordance with established ethical standards. For our experiments rely exclusively on publicly available datasets such as LaMP, MMLU, and BBH and use them strictly in accordance with their intended purpose for academic research. For the LaMP tasks, which involve user data, our TITOK framework aligns with ethical considerations by minimizing data dependence. It does not store or expose raw user data and only updates a small set of task and user-specific parameters. This helps safeguard privacy while enabling efficient knowledge transfer.
REPRODUCIBILITY STATEMENT
We provide comprehensive description of our implementation Section 4, including pipeline con- figurations, hyperparameters, models, datasets, and evaluation metrics. The source code of TrTok publicly available at https: /ttp:::///iiee /ttpttgtttt/tt////////e www.coo////////e
ACKNOWLEDGMENTS
This research was supported part by Institute for Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. RS-2020- 11201361, Artificial Intelligence Graduate School Program (Yonsei University): No. RS-2025-
25442405, Development of Self-Learning World Model-Based AGI System for Hyperspectral Imaging).
REFERENCES
Rambod Azimi, Rishav Rishav, Marek Teichmann, and Samira Ebrahimi Kahou. Kd-lora: A hybrid approach efficient fine-tuning with lora and knowledge distillation. arXiv preprint arXiv:2410.2077,, 2024.
Melis llayda Bal, Volkan Cevher, and Michael Muchlebach. Eslm: Risk-averse selective language modeling for efficient pretraining, 2025. URL bttps:://axxiy www.xiivorrg/oo.o.... aba/2/bb//255// 19893.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https: arxii..v... org/ abs /abos/2000 2005 14165.
Hyungjune Bu, Chanjoo Jung, Minjae Kang, and Jaehyung Kim. Personalized Ilm decoding via contrasting personal preference, 2025. URL https: /aaxiiv////ivvv.///aa//// www./o/gg//aa.a....ooo.o..e hataaabb/255... 2200 12109.
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training better alpaca with fewer data, 2024a. URL https: /tt////xtitt arxiv org/ abs
Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, et al. Alpagasus: Training a better alpaca with fewer data. In International Conference on Learning Representations (ICLR), 2024b.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In Advances in Neural Informtion Processing Systems (NeurIPS), 2023.
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Fos- ter, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, h, Haonan Li, Kyle McDonell, Niklas Muen- nighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf. Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. URL https: /wweeooo.... ooaeddoorrg//eee hwtp://ecordsssoe 000006/0//2600000
11

---

Published as a conference paper at ICLR 2026
Yuxian Gu, Zhengyan Zhang. Xiaozhi Wang, Zhiyuan Liu, and Maosong Sun. Train evil: Se- lective masking for task-guided pre-training. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.). Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pp. 1966,6974, Online, November 2020. Association for Computational Linguistics. doi: 10. 1 2020. emn main. waaan.566// 566/.
/ttp:///caaaatthhoooooooo ogy org/
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Ja- cob Steinhardt. Measuring massive multitask language understanding, 2021. URL https:
03300.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in neural network. arXiv preprint arXiv:1503.0253: 2015.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. International Conference on Learning Representations (ICLR), 2020.
Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pp. 2790-2799. PMLR, 2019.
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. URL https: // //arxiv. org. abs / 2106.116.... 09685.
Seungone Kim, Juyoung Suk, Xiang Yue, Vijay Viswanathan, Seongyun Lee, Yizhong Wang, Kiril Gashteovski, Carolin Lawrence, Sean Welleck, and Graham Neubig. Evaluating language models as synthetic data generators. In Annual Meeting of the Association for Computational Linguistics (ACL), 2025.
Varun Kumar, Ashutosh Choudhary, and Eunah Cho. Data augmentation using pre-trained trans- former models, 2021. URL
02245.
J. Li and S. Babu. A Graduate Course on Statistical Inference. Springer, 2019.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arxiv:2101 101.00/9000... 2021.
Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large lan- guage models for text classification: Potential and limitations, 2023. URL wwt.t:ttt/s//iiiieeeee org/abs aaab///10000 2210..1... 07849.
Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, and Weizhu Chen. Rho-l: Not tokens are what you need, 2025. URL aattps://azee
org/ ab4/2404 07965.
He Liu, Yikai Wang. Huaping Liu, Fuchun Sun, and Anbang Yao. Small scale data-free knowledge distillation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Gaurav Kumar Nayak, Konda Reddy Mopuri, Vaisakh Shaj, Venkatesh Babu Radhakrishnan, and Anirban Chakraborty. Zero-shot knowledge distillation deep networks. In Proceedings of the International Conference Machine Learning (ICML), 2019.
Jerzy Neyman and Egon S. Pearson. On the problem of the most efficient tests of statistical hy- potheses. Philosophical Transactions of the Royal Society of London. Series A, 231:289-337, 1933.
OpenAl, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapir Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann,
12

---

Published as a conference paper at ICLR 2026
Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang. Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, łukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim. Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini. Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil. David Medina, Aalok Mchta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa. Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O' Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Rousse7, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Kyla Sheppard, Toki Sherbakov. Jessica Shich. Sarah Shoker, Pranav Shyam. Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang. Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https: /ttp/v..ivvv/..///////.e./...av. org/ abs/ 22033333..2 08774.
Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Ev- timov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. Code llama: Open foundation models for code, 2024. URL https:tt/////. /atl://azzae arxiv. org/ abs/2303 2200. 12950.
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large lan- guage models meet personalization, 2024. URL https: //ta////aaiii/ www.aawwwww..a..ra..a..a.aoeooer harg//bbl aa///b////// 22044 11406. 6.
Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V. Lc, Ed H. Chi, Denny Zhou, and Jason Wei. Challenging big- bench tasks and whether thhon-of- can solve them, 2022. URL /tattts://axiv.... org/ abs/ 2210.21...22 09261.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https: ///arvi.... org//bs/ /700 03762.
13

---

Published as a conference paper at ICLR 2026
Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi. Knowledge fusion of large language models, 2024. URL https: /ttp:s///aavvvviaiaa http.//rr/abb//2..1044224
Runqian Wang, Soumya Ghosh, David Cox, Diego Antognini, Aude Oliva, Rogerio Feris, and Leonid Karlinsky. Trans-lora: towards data-free transferable parameter efficient finetuning. In Advances in Neural Information Processing Systems (NeurIPS), 2024.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct Aligning language models with self-generated instructions. In Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
Haoyi Xiong, Jiang Bian, Yuchen Li, Xuhong Li, Mengnan Du, Shuaiqiang Wang, Dawei Yin, and Sumi Helal. When search engine services meet large language models: Visions and challenges, 2024. URL https:::: wtw.v..ivvv/..///////...//..aa org/ abs 22002
Seo Yeongbin, Dongha Lee, and Jinyoung Yeo. Train-attention: Meta-learning where focus in continual knowledge learning. Advances in Neural Information Processing Systems, 37:58284- 58308, 2024.
14

---

Published as a conference paper at ICLR 2026
select Top k% tokens
But
seek
his
kingdom and his
righteousness.
x
1 kingdom and his t
0 righteousness
But
seek
first
his
Source model
But
seek
first
his
kingdom and
his righteousness
Target model
t
one
many
to
1/1-1
(1+12/
1/1-1
2/3
1
0
1
1
1
0
0
0.667
0.667
But
seek
first
his
kingdom and his
righteousness
keep Top k% tokens
Target model (+LoRA) trains on aligned top k% tokens
Figure 4: Overview of TiTok's tokenizer alignment algorithm. The algorithm handles cases where the source and target models use different tokenizers. The binary mask scores assigned by the source model averaged within aligned spans and propagated to target tokens, producing fractional scores that guide top-k% token selection for training the target model's LoRA adapter.
A TOKENIZER ALIGNMENT ALGORITHM
When the source and target models use different tokenizers, direct token-level transfer not pos- sible. To address this, implement dual-pointe alignment procedure. The algorithm maintains two pointers, one for the source tokens and one for the target tokens. At each step, the source pointer advances by one token, accumulating decoded segment, while the target pointer incrementally tends its own segment until the normalized texts match. Once match is found, the corresponding spans are recorded alignment, we apply masking rules to propagate the source binary mask scores (values that indicate whether token should be kept or discarded) the target tokens. Specifically:
alignment, and the target pointer jumps forward to that position. After
1) One-to-One: binary mask score directly copied.
2) One-to-Many: score is replicated across all aligned target tokens.
3) Many-to-One: averaged scores of multiple source tokens are assigned to the target token.
4) Many-to-Many: the averaged score of the source tokens is assigned to the corresponding aligned target tokens.
15

---

Published as a conference paper at ICLR 2026
Table 6: Average alignment percentage breakdown across BBH and News Headline tasks in the Mistral 7B - Llama3 8B transfer setting.
BBH
News Headline
exceptions
0%
0%
many to many
18%
5.00%
many
3.47%
17.45%
one to many
0.01%
0.01 %
one to one
50.33%
77.52%
Table 7: Effect of token alignment cases in the Mistral-7B - -lama-3 8B transfer experiments.
BBH
News Headline
Acc.
R-L
Vanilla
0.397
0.117
0.101
Mistral-7B + Llama-3 8B
One to one only
0.472
0.138
0.120
TiToK (k=70%)
0.482
0.160
0.142
This process yields fractional mask scores that capture the relative importance of each target token. Finally, we keep only the top-k% of tokens according to these scores, producing a final binary selection mask over the target tokens that preserves the most informative regions while discarding less relevant ones. An overview of this tokenizer alignment algorithm is presented in Figure 4.
Robustness of the algorithm. The algorithm is conceptually error-free since it simply performs deterministic mapping between two tokenizers on the exact same text sequence. Since both to- kenizations correspond
the identical underlying character string, the mapping cannot introduce semantic errors; every token in both tokenizers is defined over non-overlapping spans of the same text, and these spans align uniquely. For instance, the Mistral + Llama3 8B transfer setting on BBH and News Headline benchmarks, 100% of tokens are aligned, and there are no exception cases as shown in Table 6, confirming that alignment errors do not occur in practice.
Furthermore, additionally construct degraded setting in which aligned token pairs and discard all other alignment cases. We perform the experiment in the Mistral 7B Llama3 8B transfer setting BBH and News Headline tasks, and the full results presented in Table From these empirical results, we can observe that using all the full alignment cases outperforms using one-to-one token pairs only. This implies that the many-to-one, one-to-many, and many-to-many alignments are also correctly aligned, as including them yields the best results. Together, these results confirm that alignment method is both accurate and robust.
keep only the one-to-one
B USAGE OF AI ASSISTANTS
AI assistants minimally used in this work, restricted solely to language refinement, such as grammar correction, punctuation, and sentence structure. All research ideas, methodologies, and analyses the original contributions of the authors. Thus, the use of Al is confined to editorial support and did not influence the originality or intellectual contributions the work.
C DETAILS OF DATASETS
C.1 BIG-BENCH HARD (BBH)
Big-Bench Hard (BBH) (Suzgun et al., 2022) is designed rigorous benchmark for evaluat- ing model performance on challenging reasoning problems, including multi-step logical reasoning, symbolic manipulation, and commonsense inference. The tasks formatted multiple choice questions. Since BBH is originally a test-only benchmark, we split 90% of the data for training the source expert model and reserved 10% for evaluation. The 27 BBH tasks categorized in Table 8.
16

---

Published as a conference paper at ICLR 2026
C.2 MASSIVE MULTITASK LANGUAGE UNDERSTANDING (MMLU)
Massive Multitask Language Understanding (MMLU) (Hendrycks et al., 2021) is comprehensive benchmark for evaluating model performance across broad range of knowledge intensive tasks. The benchmark consists of multiple choice questions and, similarly to BBH, we also split the 100 randomly sampled test-only instances into 90%/10% All the 57 subtasks are categorized in Table 9.
Table Categorization of the 27 Big-Bench Hard (BBH) tasks.
Category
Tasks
Logical Reasoning
Noolean.expressione causal_judgement, date undeerstanddn.
disambiguation _qa dyck. SyyckJJnggages formal fallaais, logical deduction three objects, logical deduction five five objects, seccuction.sceen.eee
logical deduction seven objects, temporal sequences, tracking shuffled objects three objects, tracking shuffled objects five objects, web_of_lies
tracking.shai wow.:/edddddeeccsssseeeeee
Linguistic
hyperbaton, ruin names, www.ssaaationeeooorooooooooooe word_sorting
snarks,
Mathematical Symbolic
scometric_shapes, multistep_ariee wwwttarmttc.ttwo wobp:/ettttttoo/ttt.toooooooo.. object.counting.
reasoning ecasoning.about.com www.tt.boolbbo...o www.oooooooooooeeee _objects
Applied/ Knowledge
novic.recommennation, navigate, penguins_in.a table,, sports, sports understanding
Table 9: Categorization of the 57 MMLU tasks.
Category
Tasks
STEM
abstract algebra, anatomy, astronomy, college biology, college chemistry, college computer science, college mathematics, college medicine.
college.physics. computer.securfe
conceptual.physies..
electrical.engineerring, elementary
elementarrymmathhmaatee mathhmatiics, fogial.ogii,
high_school bcoooloooo.bbllogy....oooooo high www.ssschhhhchstttyyy.l _chemistry, high_school.computer....m computer_sciencee
highhschhol_mathemattoss .mathematics, high_school.physiec high_school_se _statistics
machine macmine..earming, medical.gened genettcs,, nutrition, professionalmmediiinee medicine, professional professionallpsycholoyyy. virology, clinical knowledge, human_aging human -exualityx
Humanities
business. ethics, formal. fallacies, jurisprudence, logical fallacies, philosophy, prehistory, world religions, moral disputes, moral scenarios, professional law, professional accounting. high_school.. htchssscooooeuopppeeeeeeeeeee eeroppean history, global. facts, security_studie
www.schooo..wold.hhootoooo high.school.us history,
Social Sciences
econometries, high_schon maaiencommmmnnecoomiccc high_school gaaghsshool_micreeooeee mmicroeconomics, management, marketing, public relations, sociology, foreign policy, high school geography, high school government and politics
Other
miscellaneous, global facts
C.3 LAMP TASKS
We utilize the LaMP benchmark to evaluate whether TrTok is also effective in the personaliza- tion setting. Among the LaMP tasks, we focus on the text generation tasks that suitable, accessible, and evaluable in our setting:
News Headline Generation (LaMP 4). Given author profile consisting of previously written headlines, the model is asked to generate a headline for news article. The task evaluates if the model can adapt its output reflect the author's characteristic style in journalistic writing.
Scholarly Title Generation (LaMP 5). Using an author profile built from prior titles of aca- demic publications, the model generates title for new given abstract. The task assesses the model's s ability to capture and reproduce distinctive conventions of academic writing.
17

---

Published as a conference paper at ICLR 2026
D BASELINE DETAILS
To evaluate the effectiveness of TITOK, we compare against several baselines follows:
D.I VANILLA
The vanilla baseline corresponds to the standard target base model without any additional training or knowledge transfer from the source model. This setup reflects the performance of the target model in its raw initialized state and serves as a lower bound for evaluating transfer methods. Intuitively, achieving performance that surpasses this baseline provides clear evidence that the target model has successfully acquired and internalized knowledge transferred from the source model.
D.2 KNOWLEDGE DISTILLATION (KD) (+MINED)
The knowledge distillation (KD) (Hinton et al., 2015; Azimi et al., 2024) baseline trains the student model to mimic the teacher model's output distribution. Specifically, the loss is weighted sum of the cross entropy objective and the KL divergence between the teacher and student distributions. In setup, the KD experiments conducted using the TransLoRA filtered synthetic datasets.
For cases where the source and target models use mismatched tokenizers when performing KD, apply the Minimal Edit Distance (MinED) alignment method (Wan et al., 2024). In particular, MinED matches tokens across different vocabularies by minimizing the number of character level edits. For example, can align "get" with "gets," "color" with "colour," or "analysis" with "analyses." " This tokenizer alignment approach avoids degeneracy issues in token alignment when conducting KD, though is different from our dual muiltpoitter based alignment algorithm.
D.3 TRANSLORA
The TransLoRA baseline (Wang al., 2024) transfers LoRA knowledge by generating synthetic data. In this method, the vanilla target model synthesizes queries, and the source model with its LoRA adapter provides the corresponding labels. Subsequently, discriminator, separately trained with the source model as its base, is applied to filter the synthetic data used for fine-tuning the target model's LoRA adapter. For the sake of consistency and fair comparison, generate synthetic data both from the source model and use the same hyperparameter settings in our experiments.
Comparison with ToAA TransLoRA's
assumption is transfer entire synthetic quences generated by the teacher. Therefore, its framework is tightly coupled to synthetic data and unable to function in the absence teacher-generated labels. In that sense, this also implies that TransLoRA cannot be applied to external datasets. Thus, it mainly highlights the role of synthetic data, giving less thought to the architecture of the knowledge transfer procedure itself.
In contrast, TITOK directly rethinks the mechanism of knowledge transfer itself. TITOK concen- trates on determining which tokens actually contain expert-specifee knowledge (i.e., the tokens where the expert significantly deviates from the basic model) rather than copying entire synthetic sequences. Our novel token-level contrastive signal fundamentally changes how knowledge ex- tracted and aligned. Because this process does not rely teacher-generated labels. TITOK naturally extends to external, non-synthetic datasets (See Table 4) and is therefore much more generalizable.
Therefore, despite the fact that the ultimate objective of both approaches is to solve the same high-level LoRA transfer problem, which inevitably leads to certain overlapping components, the essentially different. TransLoRA focuses transferring full outputs
underlying philosophies and relies entirely on synthetic data, whereas TITOK focuses on transferring the expert's internal knowledge signals regardless of the data source, thus being more generalizable and effective.
18

---

Published as a conference paper at ICLR 2026
Table 10: Comparison with few-shot KD baseline using five seed prompts. BBH and MMLU are reported as the average accuracy across tasks, while News Headline and Scholarly Title Gener- ation tasks are evaluated using ROUGE-1 (R-1) and ROUGE-L (R-L). KD (5-shot) denotes knowl- edge distillation performed with only five few-shot examples. Best scores are in bold.
Transfer
Method
BBH
MMLU
News Headline
Scholarly Title
Acc.
Acc.
R-1
R-L
R-1
R-L
Vanilla
0.397
0.557
0.117
0.101
0.381
0.311
Mistral 7B
Mistral 7B
KD (5-shot)
0.402
0.558
0.118
0.104
0.383
0.312
TiTok (ours)
0.432
0.563
0.160
0.142
0.473
0.414
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Mistral 7B + Llama 8B
KD (5-shot)
0.470
0.477
0.126
0.111
0.446
0.379
TiTok (ours)
0.482
0.488
0.140
0.124
0.464
0.403
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Llama3 3B + Llama3 8B
KD (5-shot)
0,470
0.479
0.126
0.111
0.446
0.379
TITOK (ours)
0,509
0.475
0.127
0.113
0.457
0.392
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Llama2 7B +> Llama3 8B
KD (5-shot)
0.470
0.477
0.126
0.111
0.446
0.380
TITOK (ours)
0.510
0.479
0.140
0.122
0.461
0.404
E COMPARISON WITH ADDITIONAL BASELINE
Here, we introduce an additional baseline that leverages few-shot supervision. In practice, our synthetic data generation process is seeded with small set of prompts, which be regarded as few-shot data. To be thorough, we therefore establish a baseline trained only on these five seed prompts, referred to KD (5-shot), that our evaluation of TITOK also considers minimal few-shot supervision. The results of this additional baseline are presented in Table 10.
Across all transfer settings, KD on only 5-shot samples provides only marginal improvements over the vamilla model, with average gains of 0.9% on reasoning tasks (BBH and MMLU) and 0.6-1.2% on personalization tasks (News Headline and Scholarly Title Generation). In contrast, TIToK delivers substantial improvements over this baseline, achieving 4.7% and 2. 2.1% average gains on reasoning tasks and 8.9-16.6% average gains on personalization tasks. Winning over KD with only 5 few-shot samples shows that TITOK is not simply the result of few-shot effect. Instead, the five seed prompts act only as a starting point, which our framework expands into richer and more effective synthetic training signals. This finding confirms that the true strength of TIToK lies in how effectively leverages limited supervision rather than in the few-shot data itself.
F EXTENDED TABLE 1 RESULTS WITH VARIANCE AND STATISTICAL DETAIL
We additionally report the mean standard deviation over random seeds of our results in Table 1. The results are presented in Table 11. As shown, the overall trend remains consistent: relative to the target vanilla baseline, KD improves by 8.5%, TransLoRA by 4.44%, and TrToK by 9.94%. In fact. these improvements remain substantially larger than the corresponding standard deviations
G COMPREHENSIVE K% SENSITIVITY ANALYSIS
In this section, we conduct an extensive experiment on TITOK's performance throughout a wide range of k values, from 10% to 90%. The results are comprehensively presented in Table 12. These experiments demonstrate that TITOK consistently outperforms the target vanilla baseline throughout broad, steady range of values, while the exact performance varies slightly depending on k%. This clearly indicates that the method is not overly sensitive to the choice of k% and that there exists a broad range of reasonable k% values where improvements reliably obtained.
19

---

Published as a conference paper at ICLR 2026
Table II: Main results with variance. Mean + standard deviation over three random seeds on BBH, MMLU, News Headline, and Scholarly Title under four transfer settings. BBH and MMLU are evaluated by exact-match accuracy, while News Headline and Scholarly Title are evaluated by ROUGE-I/L. All evaluations are zero-shot. Best scores are in bold, while second highest are underlined.
BBH
MMLU
News Headline
Scholarly Title
Transfer
Method
Acc.
Acc.
R-1
R-L
R-1
R-L
Vanilla
0.397
0.557
0.117
0,101
0.381
0
Mistral 7B
KD
0.560
0.117 1 0.001
0.104
0.003
0.385
0.006
0.310
0.005
Misiral 7B
Transt oRA
0,416
0.006
0.534
0.156
0.002
0.137
0.001
0.447
0.001
0.382
TrTox (ours, k=70%)
0.424 0,008
0.561
+: 0,002
0.161
+: 0.001
0.143
0.001
0.473 1:0
0,413 :: 0.001
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Mistral 7B
KD
0.475 0.004
0.482 + 0.004
0.127 +
0.112 + 0.001
0.454
0.001
0.387 +: 0.002
laama 8B
TransLoRA
0.003
0.473
0.126 + 0.003
0.110
0.002
0.001
0.397
TrTox (ours, k=70%)
0.484 0,002
0.485
0.003
0.139
0:001
0.123
0.001
0.464
0.001
0.403
Vanilla
0.469
0.469
0.125
0.110
0.444
0.378
Llama3 3B
KD
0.474 0.005
0.477 + 0.002
0.125 + 0.001
0,110
0.001
0.449
0.001
0.383 + 0.001
Llama3
TransLoRA
0.471 + 0,009
0.467 1: 0.002
0,122 1: 0.001
0. 108
0.001
0.454
0.001
0.387 :: 0.001
TrTox (ours, k=30%)
0.496 0.011
0.478
0.004
0.127 1 0.001
0.113
0.001
0.001 0.456
0.392 1:
Vanilla
0.469
0.469
0.110
0.444
0.378
Lluma2
KD
0.473
0.476
0.125
0.110 0.001
0.449
0.382
+9 Llana3
TransLoRA
0.472
0.002
0.468
0.002
0.123 1 0.001
0.109 0.001
0:457
0.003
0.394
0.005
TrTox (ours, k=70%)
0.488 0.019
0.477 + 0.003
0,138 4: 0.002
0.120 0.002
0.461
0.001
0.403 +: 0.001
Table 12: TYTOK performance across A = 10%-90% "Vanilla" denotes performance of target vanilla model. Highlighted columns denote the selected universal 4% reported Table 1. Highest performance are in bold, while the second best results are underlined.
Transfer
Task
Metric
Vanilla
10%
20%
30%
40%
50%
60%
70%
%0%
90%
BBII
Acc
0.397
0.401
0.420
0.444
0.445
0.441
0.432
0.432
0.431
0.431
MMLU
Acc
0.557
0.556
0.556
0.558
0.553
0.554
0.560
0.563
0.560
0.558
Mistral 7B - Mistral 7B
News Headline
0.117
0.153
0.159
0.161
0.160
0.160
0.160
0.160
R-L
0.101
0.138
0.143
0.144
0.143
0.142
0.142
0.142
0.142
0.142
Scholarly Title
R-1
0,381
0.466
0.475
0.481
0.474
0.473
0.473
0.473
0.473
0.470
R-L
0.311
0.408
0.419
0.424
0.416
0.414
0.414
0.414
0.414
0.411
BBII MMLU
Acc
0.469
0.470
0.482
0.471
0.473
0.475
0.476
0.482
0.483
0.478
Acc
0.469
0.487
0.483
0.500
0.492
0.492
0.488
0.488
0.494
0.500
Mistral 7B + Llama3 8B
News Headline
R-1
0.125
0.140
0.141
0.142
0.142
0.141
0.140
0.140
0.138
0.138
R-L
0.110
0.123
0.124
0.125
0.126
0.124
0.123
0.124
0.122
0.121
R-1
0.444
0.460
0.458
0.458
0,460
0.467
0.466
0.464
0.465
0.465
Scholarly Tille
R-L
0.378
0.398
0.394
0.395
0.396
0.406
0.405
0.403
0,403
0.406
0.515
0.507
0.512
0.492
MMLU
Acc
0.469
0.479
0.477
0.475
0.475
0.475
0.474
0.475
0.474
0.472
R-1
0.125
0.127
0.127
0.127
0.128
0.122
0.121
0.122
0.121
0.121
Llama3 3B Llama3
News Headline
R-L.
0.110
0.111
0.112
0.113
0.113
0.108
0.107
0.107
0.107
0.107
0.449
0.456
0.457
0.460
0.462
0.462
0.461
0.462
Scholarly Tille
R-L
0.378
0.385
0.390
0.392
0.397
0.400
0.400
0.398
0.398
0.397
BBH MMLU
Acc
0.469
0.527
0.519
0.531
0.510
0.514
0.514
0.510
0.509
0.509
Acc
0.474
0.481
0.482
0.477
R-1
0.125
0.126
0.129
0.130
0.135
0.138
0.138
0.140
0.139
0.139
Llama2 7B + Llama3 8B
News Headline
R-I.
0.110
0,110
0.114
0,115
0.119
0.121
0.120
0.122
0.121
0.121
Scholarly Title
R-1
0.444
0.450
0.455
0.453
0.453
0.458
0.460
0.461
0.461
0.464
R-L
0.378
0.385
0.391
0.389
0.389
0.396
0.402
0.404
0.404
0.406
The reason for choosing universal k% hyperparameter in our study is to keep the presentation of our paper more coherent and consistent. For this reason, we intentionally avoid task-specific tuning and report the k% that generally works well across tasks. However, per-task optimization of k%, further performance improvements are indeed possible.
note that with additional
With regard to the adaptive mechanism, each transfer setting exhibits its own effective k% range. For instance, same- backbbe BBH transfer favors a mid-range k% (30-70%), weak-to-strooe BBH transfer benefits from smaller k%, and weak-to-stme to-strong News Headline Generation shows the oppo-
20

---

Published as a conference paper at ICLR 2026
site trend, preferring larger k%. Taken together, these trends suggest that simple adaptive mecha- nism can already be practical and effective. In our experiments, even coarse adjustments (i.e., using around 30% for weak-to- strong reasoning transfers and 70% for weak-to-strong stylistic tasks as denoted in Section 4.3) prove sufficient, without requiring extensive hyperparameter searches.
H SYNTHETIC DATA GENERATION WITH 2x POOL AND TOP-M SELECTION
In this section, we provide further details regarding the number of synthetic data samples used. In generating synthetic data, we first provide five samples from the original training set as few-shot exemplars to guide the model toward producing outputs in the desired style and format. For each task, we initially create a synthetic pool containing twice the number of examples used in the source model's training. This pool is then filtered using token-wise contrastive excess scores, after which we retain only the top M samples, where M equals the size of the source training set. To be specifie, set M = 225 for BBH, M 90 for MMLU, and M 200 for LaMP tasks. This procedure ensures that the target model's ' LoRA adapter is trained on a dataset comparable in scale to that of the source model, while selective filtering enhances the overall quality of the retained data.
I ROUGE-L FILTERING FOR DIVERSE SYNTHETIC QUERIES
We now proceed to provide the task lists for which we did not apply ROUGE-L filtering when generating diverse synthetic queries. In general, we use ROUGE-L filtering to encourage diversity in queries, but for the tasks listed Table 13, we only applied simple deduplication. In the case of BBH, tasks such as boolean expressions or temporal sequences already follow highly restricted and repetitive patterns, making high ROUGE-L scores inevitable. For MMLU, the only tasks without ROUGE-L filtering are the history-related subjects. This is potentially because history questions often require long passages that overlap in vocabulary, phrasing, or factual references (e.g.. recurring names, dates, events). Thus, applying strict ROUGE-L filtering in such cases made it difficult to generate the required number of synthetic queries. For all remaining BBH and MMLU tasks, as well as all LaMP tasks, applied ROUGE-L threshold of 0.7 to encourage diversity while still preserving task fidelity. Table 13 provides the tasks for which ROUGE-L filtering is not applied.
Table 13: Tasks without ROUGE-L filtering.
Category
Tasks
BBH
boolean serieannexpressions date understanddng. disambiguation.cg ceometric_shapee dirical_deduction cacriioon_tree___bb citrree.bjjett..
www.aaaettc.tttee.. navigate, object-counting penguins in a table, reasoning about colored objects,
salient translation error detection, snarks, temporal sequences, hiapingsshhffflddoohhoieep crapccttsstrre_ojeeeeee web_of_lies
MMLU
high.school_wrld wootd.. high.school.c. scc.ss.ooooluuiiiii. history high school european history
J ROBUSTNESS OF TITOK SYNTHETIC DATA QUALITY
While TIToK's primary contribution is token-level transfer and is thus not tied to synthetic data, provide detailed analysis the robustness of our method to synthetic data variations.
Robustness to low-quality synthetic data. First, we deliberately construct a suboptimal syn- thetic dataset by selecting only the 250 lowest-scoring synthetic samples, evaluated by the GPT-4o-mini grading model and prompt in (scores E {0, 1, 3, 4, 5)) (Chen et al., 2024a). All training hyperparameters and filtering processes identical to the original setup. As shown in Table 14, TITOK still substantially outperforms baselines in the Mistral 7B Mistral 7B BBH transfer setting. This shows that TrTok still effective even with low-quality synthetic data.
21

---

Published as a conference paper at ICLR 2026
Model Transfer
Method
BBH (Acc.)
Vanilla
0.397
KD
0.406
Mistral Mistral
TransLoRA
0.405
TrToK (k=70%)
0.416
Table 14: BBH exact-match performance for the Mistral 7B + Mistral 7B transfer setting using
low-quality synthetic data. TrTok (k=70%) continues outperform all baselines.
Table 15: Alternative synthetic query prompt template for BBH.
Synthetic query generation prompt for [task name] in BBH.
Generate [aask_name questions like these examples:
Example I:
(few-shot example !)
Example
(few-shot example 2)
Example 3:
(few-shot example 3)
Example 4:
(few-shot example 4)
Example 5:
(fow-shot example 5)
Example 6:
Robustness data diversity. Our synthetic data generation pipeline already incorporates dedu- plication and a ROUGE-L diversity threshold. This ensures the exclusion of exact duplicates and highly similar samples. More details are provided Appendix I.
Robustness to prompt design. We further conduct experiment with different prompt for gen- erating synthetic data. The alternative prompt template provided Table tasks in the Mistral 7B - Mistral 7B and Mistral 7B - Llama3 8B transfer settings.
test this on BBH
Table 16 shows the results of using the alternative prompt for generating synthetic data. Analytically, the results show that TITOK achieves consistent improvements, and there are no substantial differ- ences across prompt choices. This demonstrates that TITOK is not sensitive to prompt formulation, long the generated examples adhere reasonable and coherent structural pattern.
Overall, we emphasize that highly refined or meticulously selected synthetic dataset is not required for TITOK. As long as the generated samples retain a relatively cohesive task structure, TITOK is still effective and not very sensitive to the particular features of the synthetic data.
K ADDRESSING SYNTHETIC DATA QUALITY AND BIAS
We now move on to explain how our pipeline incorporates several components designed to reduce bias and prevent quality degradation in synthetic data. Specifically, achieve this by incorporating several components, including a length filter that removes malformed or low-informative samples, a loken-wise contrastive excess mechanism that selects only the most informative tokens rather than relying on the entire synthetic instances, and diversity filtering (i..., deduplication and ROUGE-L thresholding) prevent mode collapse during synthetic data generation.
22

---

Published as a conference paper at ICLR 2026
Table 16: Effect of prompt variation on TITOK on BBH in the Mistral 7B Mistral 7B and Mistral
7B + Llama3 8B settings.
Transfer
Method
Prompt
BBH (Acc.)
Vanilla
0.397
Mistral 7B Mistral 7B
original (Table 22)
0.432
TITOK (k=70%)
alternative (Table 15)
0.436
Vanilla
0.469
Mistral 7B Llama3 8B
original (Table 22)
0.482
TITOK (k=70%)
alternaive(Tablle 15)
0,485
Data Source
BBH
Scholarly Title
Gold data
2.40
3.68
Mistral 7B synthetic data
2.27
3.89
Llama3 3B synthetic data
2.01
3.74
Llama2 BB_sytthetic data
2.07
3.15
Table 17: Overall average quality ratings of gold and synthetic datasets across the BBH and Schol- arly Title Generation tasks. GPT-40-m- ni evaluates each data sample on 1-5 scale.
Additionally, we further execute external quality evaluation using separate model (GPT-40-min1), adopting robust synthetic data assessment methodology established prior work (Chen et al., 2024a). Table 17 reports the average quality scores (scale 1-5) of gold data and the synthetic data generated by multiple source models used in our experiments.
These results show that the synthetic data used in experiments is comparable in quality the gold dataset, with no evidence of harmful bias dominating the data. Importantly, TrTok performs consistently well across all synthetic sources, even when the quality varies (e.g.. 2.01 vs. 2.40 in BBH), which further indicates that TiTOK does not rely on perfectly clean or unbiased synthetic data. Taken together, these findings demonstrate that TiToK robust imperfections in synthetic data, and, once again, can be effectively used even in non-synthetic datasets as shown in Table 4.
L EXPLORATION ON SELF REFINEMENT
While TrTOK primarily focuses LoRA-to-LoRA knowledge transfer, investigate its poten- tial for self-refinement. We perform this experiment on BBH the Mistral 7B - Mistral 7B and the Mistral 7B Llama3 8B transfer settings. particular, after the initial transfer step, 1) generate synthetic data using the target LoRA that had already learned TiTok's 's transferred knowl- edge, compute token-wise contrastive excess scores on the newly generated synthetic data using the original source LoRA and the its base model, and 3) train new target LoRA with the same hyperparameters in the original setting (i.e.. k..7%). The results are summarized in Table 18.
Interestingly, the results show that the iterative step yields additional improvement, implying that iterative refining is possible. We hypothesize that this is possible because the initial transfer increases the target model's familiarity with the task and partially aligns its internal representations with the source domain. As a result, the target model generates cleaner, more coherent synthetic data with fewer irrelevant patterns. This thereby enables the second iteration of TITOK to more accurately identify domain-informatiin tokens, which improves token selection and makes excess scoring more discriminative. However, model trained on real train data. By creating increasingly better synthetic data, iteration can help narrow the gap, but it is still far from fully supervised target model trained actual data.
emphasize that iterative refining synthetic data cannot surpass target
23

---

Published as a conference paper at ICLR 2026
self transfer" applies TTToK again using synthetic data generated by the learned target LoRA after the initial transfer.
Table 18: "Initial TITOK" indicates TITOK applied once. "Iterative TITOK:
BBH (Acc.)
Mistral 7B
Mistral 7B
Vanilla
0.397
Initial TrTok
0.432
Iterative TIToK: self transfer
0.456
Target expert
0,460
Mistral Llama3 8B
Vanilla
0,469
Initial TITOK
0,482
Iterative TrTok: self transfer
0.510
Target expert
0.531
BBH
Scholarly_ Title
Model Transfer
Metric
Acc.
R-L
source Iora
0,460
0.429
0.363
Llama3 3b + Llama3 8b
target base
0.462
0.444
0.378
TrToK (k=30%)
0.509
0.457
0.392
souree lora target base
0.359
0.431
0.367
Llama2 7B - Llama3 8B
0.469
0.444
0.378
TITOK (k=70%)
0.510
0.461
0.404
Table 19: Results on BBH and Scholarly Title Generation for transfer settings where the source
LoRA is weaker than the target baseline.
M ROBUSTNESS OF TITOK UNDER WEAKER SOURCE LORAS
We now discuss scenarios where the source LoRA is weaker than the target model. While it is true that TITOK uses the source LoRA as a reference, its effectiveness is not strongly dependent on source LoRA's absolute performance. To illustrate this point more clearly, Table 19 summarizes the subsets of Table in which the source LoRA underperforms the target model's vanilla version, and presents these source LoRA scores alongside the corresponding transfer results.
As shown in the table, TITOK still consistently improves the target model even when the source LoRA is weaker than the target baseline. This shows that TIToK captures that are embedded in key tokens rather than inheriting the source LoRA's full behavior.
signals
KD-based approaches, however, highly dependent on the absolute capacity of the source LoRA. By making the target model follow the teacher's logits, a weak or poorly tuned source LoRA un- avoidably transfers certain flaws, resulting in the minimal gains supported by the results in the above table. Meanwhile, TTTOK avoids these deficiencies because it fundamentally differs in how leverages the source LoRA. Instead of relying on the source LoRA's logits globally, TITOK uses the source only to smartly identify informative token positions. As a result, a poorly tuned source LoRA does not mislead TiTok since it does not make the target model imitate the source LoRA's behavior. Rather, it selectively retrieves the useful tokens using the source LoRA's domain knowledge.
N COMPUTATIONAL OVERHEAD COMPARISON
Here, we present the computational differences between TITOK and TransLoRA. We conduct an end-to-end time comparison of TransLoRA and TITOK on the BBH and News Headline Generation benchmarks in the Mistral 7B - Llama3 8B transfer setting. The results are presented in Table 20 and 21. Across both benchmarks, TITOK achieves roughly 1.5x-2.5x reduction averagely in total compute time. Most of the gain comes from the removal of discriminator training, while the per-token ligeliiel computation produces only minimal and manageable overhead.
24

---

Published as a conference paper at ICLR 2026
BBH (250 data x 27 tasks)
Method
Metric
discriminator training
selecting data
training
TOTAL
avg.Jotal (sec/nom_rask)
279.33936
31.72915
604.61636
415
TransLoRA
avg sample (sechamplee
0.63251
0.06346
0.41847
1.11444
Method
Metric
token-wise contrastive excess scnre selecting tokens. (k 0.7)
token align
training
TOTAL
avg avg.lotal (scc/num_tas)
164,70895
0.01
0,41080
102.02867
267. 267.15997
TITOK
avg sample (scc/sample)
0.32942
0.00005
0.00054
0,40811
0.73812
Table 20: Runtime comparison for Mistral 7B Llama3 8B transfer on BBH.
News Headline (200 data 30 users)
Method
Metric
discriminstor training
selecting data
training
TOTAL
avg Jotal (sec/oumm_ueee users)
245. 10085
25,39730
58.71148
229.3
TransLoRA
wgg.sample (scc/sample)
0.61279
0,06358
0.29356
0.96993
Method
Metric
loken-wise contrastive excess selecting (.7)
token align
training
TOTAL
agg Jotal (sec/numm, users)
81.31437
0.01863
0.89304
49.5 52766768
131.75371
TYTOK
avg. wggsample (sec/sample)
0,20355
0.00005
0.00447
0.24764
0.45570
Table 21: Runtime comparison for Mistral 7B -> Llama3 8B transfer on News Headline Generation.
O QUALITATIVE EXAMPLES
In this section, provide qualitative examples from the News Headline Generation in the Mistral 7B + Mistral 7B transfer setting to illustrate how TITOK identifies the most informative tokens. The qualitative examples presented in Figure 5. Analytically, the selected tokens correspond to the beginnings major semantic or structural units in the headline. These tokens carry the highest information value: they denote transitions, introduce core phrase components, contain root words. Moreover, since the boken-wise contrastive excess score is defined as the difference between the expert model (source LoRA base) and the amateur model (base only), pinpoints the tokens where the expert and amateur diverge most in their predictions and so include effective learning signals for target tasks. Taken together, these qualitative examples effectively demonstrate that TITOK consistently retains the tokens that are most important for providing structure and meaning.
P PROMPTS FOR SYNTHETIC QUERY GENERATION
In this section, we present the prompts used for generating synthetic queries. Each prompt includes five examples, which correspond to the seed prompts taken from the original data. For BBH, which consists of multiple subtasks, we specify task task-specific instructions and formatting rules so that the generated queries and labels are structured according to the expected format. For example, Table 22 boolean.expressiien expressions as representative example in detail.
illustrates the case of boolean
In contrast, for MMLU, a single unified prompt format is sufficient for most subjects. The exceptions are the history-relater tasks, namely, sigghhsshhoolssshhstoro high history, and high school european history. For these tasks, is observed that more specific prompting is required obtain better generations. The general MMLJU prompt is shown in Table 23, whereas the history-related tasks make use of the specialized prompt provided in Table 24.
Finally, for LaMP tasks, apply the same query template uniformly to each user. The detailed prompt templates used for this benchmark are provided in Tables 25 and 26.
25

---

Published as a conference paper at ICLR 2026
You are news headline generator.
Generate headline for the following article.
Prompt
article: With scalloped hem, it's show-stopping piece that's both unique and affordable. Want more? Be sure to check out Stylelist on...
headline:
Output
Stylist": The Perfect White Dress For Your Wedding (PHOTOS) 

headline generator.
Generate headline for the following article.
Prompt
article: It's hard to imagine that you would be interested in hearing about my favorite holiday tradition, but if you are, I'd be happy share.....
headline:
Output
How To Make Your Own Christmas Ornaments (PHOTOS) 

Figure 5: Qualitatave examples from News Headline Generation in the Mistral 7B Mistral 7B setting. Bold and highlighted tokens are part of the selected tokens. Notably, these tokens exhibit high wose contrastive excess scores and thus fall into the top 70% selected for training.
26

---

Published as a conference paper at ICLR 2026
Table 22: Synthetic query prompt for the BBH boolean exoresnnessrsssions task. Each BBH subtask re- quires task-specific instructions and formatting rules to ensure that generated queries and labels follow the expected format. We show boolean expressions task here as a representative example.
Synthetic query generation prompt for BBH (boolean.eepresssitie fask.
System
You expert task generator for boolean_expression tasks. Generate boolean expression
evaluation tasks ending with is SINGLE LINE ONLY.
CRITICAL FORMAT REQUIREMENTS:
Follow the EXACT format structure shown in the examples.
ABSOLUTELY CRITICAL: Generate ONLY ONE LINE, ONE TASK per response.
-NO multiple lines, NO newlines ((n), NO multiple expressions.
Must end with is'
Example of INVALID output: True or False is)n)n False and True Example of VALID output: True or False or not False and ( True or False is
Generate diverse content but maintain the exact same format structure. Only output the task input, not the solution.
User
Generate new bool ean expressi ons tasks following these exact format examples:
Example 1: [boolean expression ending with is
Example 2: [boolean expression ending with is
Example 3: [boolean expression ending with is Example 4: [boolean expression ending with is ']
Example 5: [boolean expression ending with is
CRITICAL: Generate ONLY ONE LINE, exactly like the examples above.
Generate new task following the exact format:
27

---

Published as a conference paper at ICLR 2026
Table 23: Synthetic query prompt for the MMLU. Applied to all tasks except for history related tasks.
Synthetic query generation prompt for MMLU (general).
Example 1:
Question: [question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 2:
Question: [question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 3:
Question: [question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 4:
Question: [question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 5:
Question: [question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
[choice snippet]
Answer: [1-4]
Example 6:
28

---

Published as a conference paper at ICLR 2026
Table 24:
Synthetic query prompt for the MMLU history tasks only. The history tasks are high school us history, high school world history, and high school european history. These his- tory tasks required more specific prompting to obtain better results.
Synthetic query generation prompt for MMLU (history related tasks).
Generate [SUBJECT] multiple choice questions following these examples:
Example I:
Question: [history question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 2:
Question: [history question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
[choice snippet]
Answer: [1-4]
Example 3:
Question: [history question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 4:
Question: [history question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Example 5:
Question: [history question snippet]
1. [choice snippet]
2. [choice snippet]
3. [choice snippet]
4. [choice snippet]
Answer: [1-4]
Now generate a new [SUBJECT] question following the same format:
Question:
29

---

Published as a conference paper at ICLR 2026
Table 25: Synthetic query prompt for the News Headline Generation task.
Synthetic query generation prompt for News Headline Generation
You are news text generator. Generate diverse news article texts that could be used to create headlines. Only output the raw news text content, not headlines or queries.
Example [article snippet]
Example 2:
[article snippet]
Example 3: [article snippet]
Example 4: [article snippet]
Example 5:
[article snippet]
Example 6:
Table 26: Synthetic query prompt for the Scholarly Title Generation task.
Synthetic query generation prompt for Scholarly Title Generation
You are scholarly abstract generator. Generate diverse ONE paragraph abstracts that ask for creating paper titles from abstracts. The abstract should be ONE paragraph only. Only output the raw abstract text, not the actual titles.
Example 1: Create title for this research paper:
Abstract: "[abstract snippet]" Title: "[title]"
Example 2: Create title for this research paper: Abstract: "[abstract snippet]" Title: "[title]"
Example 3: Create title for this research paper: Abstract: "[abstract snippet]"
Title: "[title]"
Example 4: Create title for this research paper: Abstract: "[abstract snippet]"
Title: "[title]"
Example 5: Create a title for this research paper: Abstract: "[abstract snippet]"
Title: "[title]"
Write research paper abstract as single paragraph containing at least 3 sentences.
Example 6:
30


Reviews:
[{"review_id": "NQV3STp173", "reviewer": "Reviewer_AmyZ", "summary": "This paper introduces TITOK (Transfer Token-level Knowledge), a framework for transplanting LoRA adapters across different LLM backbones by transferring token-level knowledge instead of sequence-level knowledge.\nUnlike previous approaches such as TransLoRA, which require a discriminator model to filter synthetic data, TITOK leverages a contrastive excess signal to identify informative tokens that encode task-specific knowledge. TITOK requires neither extra models nor additional training overhead. It comes with an effective mechanism to resolve tokenizer mismatches between source and target models, which enhances robustness and applicability.", "questions": "- How sensitive is TITOK to the quality of the source LoRA? Would a poorly tuned LoRA produce misleading excess scores?\n- Could contrastive excess be used iteratively (e.g., to refine the LoRA itself) rather than just for one-shot transfer?\n- How does TITOK behave when transferring between models with different architectures (e.g., dense vs. mixture-of-experts backbones)?\n- Can the authors clarify the computational overhead of computing per-token log-likelihood differences compared to TransLoRA’s discriminator?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The idea of contrastive excess as a proxy for task-specific token importance is elegant and avoids the need for auxiliary discriminators or extra training modules. It’s conceptually sound and efficiently leverages existing model behavior.\n- TITOK uses no extra models and operates with modest computational overhead compared to TransLoRA. This design aligns well with real-world deployment constraints.\n- The authors test across multiple model families (Mistral, Llama 2/3) and tasks (reasoning, personalization), presenting clear gains in each. The inclusion of ablations and sensitivity analyses (on token selection ratio, query generation source, etc.) adds strong empirical credibility.\n- The dual-pointer alignment and averaging rules are simple yet effective.", "weaknesses": "- The proposed method still relies on generating a large pool of synthetic samples, which may introduce significant costs\n- While token-level filtering is the paper’s key innovation, there is no qualitative or visual analysis demonstrating which tokens are retained or why they contribute to effective transfer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hG139fdx1l", "reviewer": "Reviewer_LDW6", "summary": "This paper presents a framework called TITOK for transferring LoRA-based knowledge between large language models (LLMs). The authors note that existing parameter-efficient fine-tuning (PEFT) methods such as LoRA cannot be directly transplanted across different backbones, while knowledge distillation (KD) approaches depend heavily on access to original task data. TITOK introduces the concept of contrastive excess, computed between a source model with LoRA and its corresponding base model without LoRA, to identify task-informative tokens. Using these token-level signals, the method filters synthetic data and trains the target model’s new LoRA adapter only on the most informative tokens, avoiding the need for an additional discriminator or real training data. Experiments on BBH, MMLU, and LaMP benchmarks show that TITOK consistently outperforms KD and TransLoRA across multiple transfer settings, achieving average performance gains of 4–8%. Overall, the paper proposes a lightweight token-level transfer framework and demonstrates its effectiveness through empirical evaluation.", "questions": "1. The paper lacks a clear explanation of why the contrastive excess between the LoRA-equipped and base models effectively captures transferable task knowledge. Could the authors provide theoretical reasoning or empirical evidence (e.g., gradient or information analysis) to support its validity?\n\n2. TITOK heavily depends on synthetic data generated by the source model. How robust is the method to variations in data quality, prompt design, or model strength? Would TITOK still perform well if the source model or synthetic data are suboptimal?\n\n3. The algorithm’s performance appears sensitive to the top-k% token selection and sample filtering. Can the authors provide quantitative sensitivity analyses or propose adaptive mechanisms to make TITOK more stable across tasks?\n\n4. The tokenizer alignment process is crucial but not quantitatively evaluated. How accurate is this alignment across model families with different tokenization schemes? Could small alignment errors lead to degraded performance?\n\n5. The comparisons only include KD and TransLoRA. To better position the contribution, could the authors include or discuss results against more recent PEFT transfer methods (e.g., KD-LoRA, AdapterFusion, DoRA)? This would clarify TITOK’s true advantage.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper introduces a new perspective on LoRA knowledge transfer by focusing on token-level contrastive signals (“contrastive excess”) rather than sequence-level alignment. This provides a fine-grained and model-agnostic approach that does not require additional networks or data annotations.\n2. The authors conduct systematic experiments across multiple benchmarks (BBH, MMLU, and LaMP) and various transfer settings (intra-family, cross-family, and cross-version). The results consistently demonstrate that TITOK achieves stable improvements over established baselines such as KD and TransLoRA.", "weaknesses": "1. The paper does not provide a solid theoretical justification for why the proposed contrastive excess signal effectively captures transferable task knowledge. The formulation remains largely heuristic, and no analysis is presented to support its generalization properties or convergence behavior.\n\n2. Although the experiments cover multiple datasets, the reported improvements (4–8%) are modest and sometimes within the margin of variance. The absence of standard deviations or statistical significance tests weakens the credibility of the claimed gains.\n\n3. Despite claiming to reduce data dependence, TITOK still relies heavily on synthetic data generated from the source model. The paper does not adequately evaluate how data quality, diversity, or prompt design affect transfer performance, making reproducibility and robustness questionable.\n\n4. The paper mainly compares against KD and TransLoRA but omits other recent or stronger PEFT transfer approaches. Without these comparisons, it is unclear whether TITOK’s benefits are meaningful beyond this narrow baseline set.\n\n5. Several key implementation details—such as the choice of hyperparameters (e.g., k% token ratio) and the impact of tokenizer alignment—are only described qualitatively. The algorithmic intuition is under-explained, and the overall presentation lacks ablation depth, which limits interpretability and reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lynnvkiRbS", "reviewer": "Reviewer_YLn5", "summary": "This paper proposes TITOK, a lightweight framework for transferring LoRA adapters across LLMs without original training data. It uses token-level contrastive excess scores from the difference between a base model and its LoRA version to filter synthetic data and focus training on informative tokens. TITOK avoids extra discriminators or full datasets, achieving consistent improvements over baselines like KD and TransLoRA across tasks and model types.", "questions": "Did the authors perform any human evaluation or external fact-checking to verify the quality of the filtered synthetic data?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. TITOK avoids extra models or full datasets by leveraging only token-level contrastive excess scores, making it computationally efficient and easy to deploy.\n\n2. Demonstrates consistent performance gains across diverse transfer settings, including different model families, sizes, and tokenizers, highlighting robustness and broad applicability.", "weaknesses": "- Only 7–8 B models; no evidence it scales to 70 B+ or harder generative tasks.\n\n- Source model generates both query and label; filter can’t catch wrong or biased synthetic data.\n\n- No formal justification that “contrastive excess” equals transferable knowledge.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UQSkLVq5Pd", "reviewer": "Reviewer_g2oB", "summary": "This paper introduces TITOK, a framework for transferring LoRA adapters between large language models through token-level contrastive knowledge transfer.\nUnlike existing methods such as TransLoRA, which rely on synthetic data filtered by an external discriminator, TITOK uses a self-contained contrastive excess mechanism that compares token-level likelihoods between a source model with and without LoRA. Tokens where the LoRA contributes the most (“excessive” tokens) are identified as carrying rich, task-specific knowledge.", "questions": "See weaknesses", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Lightweight and scalable.\nTITOK requires no additional model training; it simply reuses the source model’s log-likelihoods. The proposed sample filtering and token selection can be implemented efficiently and applied to diverse architectures.\n\n2. Cross-tokenizer alignment.\nThe tokenizer-alignment algorithm (Fig. 4) is an important practical contribution that allows reliable transfer across families like Mistral→LLaMA, addressing a frequent but rarely discussed issue in adapter transplantation.\n\n3. Comprehensive evaluation.\nThe experiments cover both reasoning (BBH, MMLU) and personalization (LaMP) tasks, across intra- and inter-family transfers (same model, different family, size, and version). Ablations, sensitivity to token selection ratio, and external-data experiments are thorough and convincing.\n\n4. Strong empirical results.\nTITOK consistently improves over baselines (e.g., +7.11% average gain in cross-model transfer) and shows robustness even when transferring with unrelated data (Table 3).", "weaknesses": "1. Incremental relative to TransLoRA.\nWhile the contrastive excess concept is novel, much of the overall framework (synthetic data, adapter transplantation) follows TransLoRA. The improvement may be viewed as a simplification rather than a fundamental rethinking of LoRA transfer.\n\n2. Limited theoretical justification.\nThe paper could benefit from more formal analysis—e.g., why token-level contrastive loss correlates with adapter informativeness, or whether excess scores approximate information gain.\n\n3. Evaluation scope.\nAll experiments use instruction-tuned LLMs up to 8B parameters. It remains unclear whether TITOK scales to larger frontier models (≥70B) or non-text modalities.\n\n4. Minor clarity issues.\nThe term “contrastive excess” could be better linked to contrastive learning literature; at present it may be confusing since no contrastive pairs or negatives are explicitly optimized.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)