Test-Time Learning for Large Language Models
Authors: Jinwu Hu
Zitian Zhang
Guohao Chen
Xutao Wen
Chao Shuai
Wei Luo
Bin Xiao
Yuanqing Li
Mingkui Tan
Venue: ICML 2025 poster (2025)
Keywords: Large Language Models
Test-Time Learning
Primary area: deep_learning->large_language_models
TL;DR: We propose a Test-Time Learning paradigm for Large Language Models that dynamically adapts them to target domains using only unlabeled test data.

Abstract: While Large Language Models (LLMs) have exhibited remarkable emergent capabilities through extensive pre-training,  they still face critical limitations in generalizing to specialized domains and handling diverse linguistic variations, known as distribution shifts. In this paper, we propose a Test-Time Learning (TTL) paradigm for LLMs, namely TLM, which dynamically adapts LLMs to target domains using only unlabeled test data during testing. Specifically, we first provide empirical evidence and theoretical insights to reveal that more accurate predictions from LLMs can be achieved by minimizing the input perplexity of the unlabeled test data. Based on this insight, we formulate the Test-Time Learning process of LLMs as input perplexity minimization, enabling self-supervised enhancement of LLM performance. Furthermore, we observe that high-perplexity samples tend to be more informative for model optimization. Accordingly, we introduce a Sample Efficient Learning Strategy that actively selects and emphasizes these high-perplexity samples for test-time updates. Lastly, to mitigate catastrophic forgetting and ensure adaptation stability, we adopt Low-Rank Adaptation (LoRA) instead of full-parameter optimization, which allows lightweight model updates while preserving more original knowledge from the model. We introduce the AdaptEval benchmark for TTL and demonstrate through experiments that TLM improves performance by at least 20% compared to original LLMs on domain knowledge adaptation.

Test-Time Learning for Large Language Models
Jinwu Hu 12*
Zitian Zhang'
Guohao Chen 12 Xutao Wen Chao Shuai
Wei Luo 42* Luo
Bin Xiao Yuanqing Li 2t Mingkui Tan 16t
Abstract
1. Introduction
While Large Language Models (LLMs) have ex- hibited remarkable emergent capabilities through extensive pre-training, they still face critical limi- tations in generalizing to specialized domains and handling diverse linguistic variations, known as distribution shifts. In this paper, we propose a Test-Time Learning (TTL) paradigm for LLMs, namely TLM, which dynamically adapts LLMs to target domains using only unlabeled test data during testing. Specifically, we first provide em- pirical evidence and theoretical insights to reveal that more accurate predictions from LLMs can be achieved by minimizing the input perplexity of the unlabeled test data. Based on this insight, we for- mulate the Test-Time Learning process of L.MSs input perplexity minimization, enabling self- supervised enhancement of LLM performance. Furthermore, we observe that high-perplen perplexity samples tend to be more informative for model optimization. Accordingly, we introduce a Sam- ple Efficient Learning Strategy that actively se- lects and emphasizes these high-perplexity sam- ples for test-time updates. Lastly, to mitigate catastrophic forgetting and ensure adaptation sta- bility, we adopt Low-Rank Adaptation (LoRA) instead of full-pate aallmearrreter optimization, which al- lows lightweight model updates while preserving more original knowledge from the model. We introduce the AdaptEval benchmark for TTL and demonstrate through experiments that TLM im- proves performance by at least 20% compared to original LLMs on domain knowledge adaptation.
Large Language Models (LLMs) such as GPT-4 (Achiam et al., 2023) and LLLaMA (Dubey al., 2024) have signif- icantly advanced the field of natural language processing (NLP), demonstrating exceptional capabilities in both un- derstanding and generating human-like text (Wang et al., 2025a). Such success is achieved through extensive pre- training on massive corpora, enabling them to learn rich lan- guage representations that facilitate superior performance in various NLP tasks (Hu al., 2025a; Zhong et al., 2024).
Despite their impressive capabilities, LLMs face significant challenges when deployed in real-world environments with dynamic and diverse data distributions. These challenges stem from the inherent sensitivity of deep learning models, including L.LMs, to distribution shifts between training and test data, often leading to substantial performance degrada- tion (Akyürek al., 2024). These distributional shifts mani- fest in two main ways: 1) Domain-Speeiffie Terminology: Encountering rare or specialized terms and structures in fields such as medicine or agriculture may limit the model's performance (Gu et al., 2021). 2) Linguistic Diversity Variations: Variations user intent and linguistic diversity, including dialects and slang, lead distributional discrep- ancies that negatively affect the model's comprehension and response generation (Bella et al., 2024).
Recently, several attempts have been proposed to improve the performance of models in dynamic and diverse real- world environments. Most existing methods can be broadly categorized into four types, as shown in Table 1. Fine- tuning (Hu al., 2022: Thirunavukarasu et al., 2023) adapts pre-trained models to specific tasks by updating their pa- rameters with labeled data, but it is constrained by the need for extensive labeled datasets, limiting its practicality in dynamic environments. Netrieval-Augmenne (RAG) (Fan et al., 2024) improves performance without requiring labeled data updates by leveraging external knowl- edge retrieved during inference, but its success depends heavily on the quality of the retrieved information. Test- Time Adaptation (TTA) (Wang et al., 2021; Niu et al., 2022a; Chen et al., 2024b) adjusts model parameters during infer- ence using only unlabeled test data, allowing the model to adapt to distribution shifts in real-time. However, most TTA
Equal contribution 'School of Software Engineering, South China University of Technology, China Pazhou Lahoratory, China "Zhejiang University, China "South China Agricultural University, China. "Chongging University of Posts and Telecommunications, China "Key Laboratory of Big Data and Intelligent Robot, Ministry of Education, China. Correspondence to: Mingkui Tan <inngkui- weee@euuuuuuuuucrrrm Yuanqing <auygli @ scut.edu.co en>, Bin Xiao <xiaooin @ cqupt.edu.cn>..
Generation
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 20255 by the author(s).

---

Test-Time Learning for Large Language Models
Table 1. Characteristics of problem settings for adapting trained models to potentially shifted test domains.
Setting
Knowledge Source Data Target Data Training Loss Testing Loss Learning Type
Fine-tuning
X
y
L(x .²)
Supervised
Retrieval-Augmente Generation (Fan et al., 2024)
X
x
Test-Time Adaptation (Wang et al., 2021)
X
X
r.
x
C(a') x""","";00
Unsupervised
Test-Time Training (Hardt& & Sun, 2024)
y
x²
x
Test-Time Learning (Ours)
X
X
x²
X
[(a*)
Self-supervised
methods rely on entropy minimization as the optimization objective, which overlooks the autoregressive dependencies within LLMs, limiting its effectiveness in improving perfor- mance on dynamic tasks (see Figure 1a). Teme-Time Training (TTT) (Hardt & Sun, 2024; Hübotter et al., 2024) retrieves data relevant to the input from the training set or knowledge base during inference to fine-tune the model, improving its performance in dynamic scenarios. However, these methods assume that the model's training data or knowledge data is accessible, which is often not the case in practice, and they also incur additional retrieval overhead.
(LoRA) (Hu et al., 2022) is more effective at mitigating catastrophic forgetting compared to full parameter updates (see Observation 3). Based on this, we utilize LoRA for TTL parameter updates, enabling lightweight training and effectively mitigating catastrophic forgetting, in contrast to updating the full parameters of LLMs. Lastly, we construct a comprehensive benchmark named AdaptEval for TTL.
We summarize our main contributions as follows:
Empirical Insights on Input Perplexity Minimiza- tion: We empirically demonstrate that output perplex- ity can be reduced by minimizing input perplexity, Based on this insight, we adopt input perplexity mini- mization the optimization objective, enabling LLMs to adapt effectively to target domains during test time.
Although recent methods address distributional shifts in test data, they still face the following limitations: 1) Difficulty in acquiring labeled data: High-quality labeled data for SFT of LLMs, especially for domain domain-specific tasks, is time- consuming, and becomes more difficult in online model up- dates. 2) Neglecting autoregressive dependencies: Many existing methods, such as TTA, overlook the autoregressive nature of LLMs, leading to potential harm when using en- tropy minimization for parameter updates. 3) High training overhead and catastrophic forgetting: Many methods re- quire substantial computational resources to update model parameters and may suffer from catastrophic forgetting.
Sample Efficient Learning Strategy with High- Perplexity Focus: We propose a Sample Efficient Learning Strategy using a perplexity -base weighting scheme to select high-perplexity test samples for up- date, ensuring efficient utilization of computational re- sources. Moreover, we use LoRA to enable lightweight training and mitigate catastrophic forgetting.
To address these limitations, we propose a Test-Time Learning (TTL) method for Large Language Models, namely TLM, which dynamically adapts L.L.Ms using only unlabeled test data. Specifically, we provide empirical evi- dence and theoretical insights to reveal that more accurate autoregressive predictions from LLMs can be achieved by minimizing the input perplexity of the unlabeled test data (see Observation 1). Based on this insight, we formulate the TTL process of LLMS as input perplexity minimization, enabling self-supervised enhancement of LLM performance. Furthermore, we observe that high-perplexity test samples contribute more significantly to model updates compared to low-perplexity samples (see Observation 2). Building on this observation, we propose a Sample Efficient Learning Strategy that employs perplexity-basee based weighting scheme to actively select and emphasize high-perplexity test samples for backpropagation, thereby facilitating efficient parameter updates during Test-Time Learning. Moreover, we observe that during the Tist-Time&a Learning, Low-Rank& Adaptation
Benchmark and Experimental Validation of Test- Time Learning: We establish the AdaptEval bench- mark for TTL and demonstrate through experiments that our TLM improves performance by at least 20% over original LLMs on domain knowledge adaptation.
2. Related Work
The adaptability of deep learning models to dynamic and di- verse real-world environments has emerged as a prominent focus in recent research. Various methods have been pro- posed to enhance model performance under distributional shifts, as summarized in Table 1.
Fine- Tine-Tuning adapts pre- trained models to specific tasks or domains by updating their parameters, such as LoRA, with labeled data (Hu et al., 2022; Thirunavukarasu et al., 2023; Chen et al., 2024c; Wang et al., 2025b). This ap- proach allows models to specialize in domain- sspccffc tasks by leveraging transfer learning to refine their capabilities.
2

---

Test-Time Learning for Large Language Models
However, it is often constrained by the need for extensive labeled datasets and high computational costs, which limit its practicality in dynamic environments where data distri- butions are continuously evolving. In contrast, our work aims to dynamically update the model at test-time using unlabeled input data, climinating the need for extensive labeled datasets and addressing the challenges posed by evolving data distributions.
is often impractieal. Unlike TTT, our focus is on TTL, which dynamically updates LLMs during test time using only unlabeled test data.
3. Problem Formulation
Without loss of generality, let P(x) denote the distribution of the training data {=+);= where x; ~ P(x). The fe- represent general Large Language Model (LLM) that has been supervised fine-tuned (SFT) on labeled training data (1a, 14) ) with parameters O During training, the model fe" (x) is optimized to generate coherent and contex- tually appropriate sequences by predicting the next token in an autoregressive manner, effectively fitting the training data and generalizing to test data from the same distribution I ~ P(x). However, in real-world deployments, the distri- bution of test data may differ significantly from the training distribution due to various factors, leading to phenomenon known as distribution shift. For general LLMs (e.g.. LLaMA and Qwen), two primary types of out-of-distribution (OOD) scenarios can occur during inference: 1) Vertical Domain Shift: This occurs when test data contains domain-specinf terminology, such as in medical, legal, or technical fields, which the model was not explicitly trained on, impairing its performance. 2) Distributional Shift in Non-Specific Do- mains: Even without a specific vertical domain, factors like user intent variations and linguistic diversity (e.g., dialects, slang) can shift test data distribution from training data, af- fecting model understanding and response generation. In these cases, the generative performance of the model fe (x) may deteriorate significantly because the model has not been explicitly trained to handle such distribution shifts, resulting in less coherent or contextually appropriate text generation on OOD test samples x ~ Q(z), where ((r) # P(æ).
Retrieval-Augmentee Generation (RAG) incorporates ex- ternal knowledge by retrieving relevant information from knowledge base during inference (Jiang et al., 2024; Qian et al., 2024; Asai et al., 2024). This allows the model to gen- erate more accurate and contextually grounded responses without requiring parameter updates. Qian et al. (2024) pro- pose MemoRAG, a retrieval-augmented generation frame- work enhanced by long-term memory for improved task performance. RAG is effective for tasks requiring up-to- date or domain knowledge but relies heavily on the quality of retrieved information and incurs additional computational latency, limiting its suitability for üime-sensitin tasks.
Test-Time Adaptation (TTA) dynamically updates model parameters during inference by utilizing unlabeled test data (Wang et al., 2021; Niu et al., 2022a; 2023; Chen et al., 2024b;a; Liang et al., 2024; Yi et al., 2024). This approach enables real-time adaptation to distributional shifts, making it suitable for scenarios where labeled data is unavailable or the test data distribution deviates significantly from the training distribution. Wang et al. (2021) propose the test entropy minimization method, which improves model con- fidence by minimizing prediction entropy through online updates of normalization statistics and affine transforma- tions. Most TTA methods rely on entropy minimization, but this approach is not well-suited for the dynamic updates required by LLMs, as shown in Figure la. To address this issue, we propose minimizing the perplexity of test samples as the optimization objective, which effectively enhances the performance of LLMs in dynamic environments.
Test-Time Learning (TTL) seeks to improve the perfor- mance of LLMS in the target domain by adjusting the model using only test data. Specifically, given a set of OOD test samples {x, M where 2, ~ (z)), the goal of TTL is to optimize the model parameters O to improve the quality and coherence of generated text for these test samples. Formally, TTL can be framed as the following optimization problem, where the objective is to minimize an unsupervised criterion defined over the test data:
Test-Time Training (TTT) retrieves relevant data from
the training set or a knowledge base during inference and uses it to fine-tune the models (Niu et al., 2022b; Hardt & Sun, 2024; Hübotter et al., 2024). This allows the model to leverage adjacent data to better adapt to current test inputs, improving its performance in dynamic scenarios. Hardt & Sun (2024) propose a test-time training approach for LLMs by fine-tuning the model on retrieved nearest neighbors from a large-scale text embedding index. Hübotter et al. (2024) propose SIFT, data selection algorithm that optimizes information gain to outperform Nearest Neighbor retrieval for test-time fine-tuning with low computational overhead. However, TTT assumes that the model's training data or knowledge base is accessible during meptoyment, and the retrieval process introduces computational overhead, which
min L(r;0), x x~ Q(x),
where θ C θ represents the subset of model parameters to be updated during the TTL. The TTL objective L(-) can take various forms, such as minimizing the perplexity. The key challenge of TTL is to design efficient adaptation strategies that can utilize unlabeled test data to improve performance on OOD samples while maintaining training efficiency and effectively mitigating catastrophic forgetting.
3

---

Test-Time Learning for Large Language Models
0.4
1.0
Geography
0.34
Low perpleiity sumples
ROUGB-LSUU -SUM
0.3
0.319
0.8
Medicine
Agriculture
ROUGE ROUGE-LSSUM 1SUM
0.32
High perplexity samples
0.245
Outgut PPL
0.6
Finance
0.2
0.4
0.1
0.078
0.2
0.26
0.0
0.0
Origin LLM
Entropy Min.
Perplexity Min.
0.0
0.2
0.4
03.6
0.8
1.0
100%
20%
40%
600%
800%
100%
Input PPL
The Proportion Samples Invalved Training
(a) Effectiveness of Entropy and Perplexity Minimization Strategies.
(b) Trends in perplexity to input and perplex- (c) Effect of different test samples in test- ity to output under Ilama3. I -nS--uutt
time perplexity minimization.
Figure 1. Summary of our exploration and observations: (a) demonstrates that perplexity minimization improves the performance of LLMs, while entropy minimization (Wang et al., 2021) may harm their performance: (b) reveals that the trend of LLM's perplexity to the input P(=) and perplexity to the output (y)[z) is the same (results are normalized). i.e., we can min P (ulla; O) by min P(2; G); and (c) 0 emphasizes that training on high-perplexity samples makes more contribution than low-perplexity ones.
4. Test-Time Learning for LLMs
Algorithm The pipeline of proposed TLM.
Input: Test samples Drest (uyر)= the trained LLM
In this paper, we propose a Test-Time Learning (TTL) method for Large Language Models (LLMs) called TLM, which dynamically adapts LLMs using only unlabeled test
fe()). LoRA AO with trainable parameters B and A, batch size B.
data. The pipeline of TLM is shown in Algorithm 1, our pro- posed TLM is composed of three key components. Inpul Perplexity Minimization Objective: Inspired by the strong correlation between input perplexity and output perplexity, we adopt input perplexity minimization as the optimization objective. This enables L.Mss to better fit the target data distribution during test time, as detailed in Sec. 4.1. 2) Sample-Efficient Learning Strategy: Not all test samples equally impact model updates. Employing perplexity- based weighting scheme, the model actively selects and emphasizes high-perplexity test samples for backpropaga- tion, thereby enabling efficient parameter updates during Test-Time Learning (c.f. Sec. 4.2). 3) Lightweight Parame- ter Updates via LoRA: To mitigate catastrophic forgetting and reduce computational costs, we integrate LoRA into TTL. By updating only small subset of model parameters, LoRA enables lightweight training and effectively mitigates catastrophic forgetting, making our proposed method suit- able for real-world deployment (c.f. Sec. 4.3).
1: Initialize LoRA parameters AQ.
Add LoRA parameters trained LLM e AO.
for a batch X' { in Test do
Calculate predictions for all X via fe()).
5: Calculate sample selection score S(x) via Eqn. (6).
6: Update LLM (Ô) with (5).
7: end for
Output: The final LLM (9).
ized by O. A lower perplexity indicates that the model's predictions more confident and closely align with the true distribution of the data, which implies better model fitting (Jumelet & Zuidema, 2023). Therefore, for a given dnesstonnanswer pair {x, y) minimizing the perplcxity of the model's response y can enhance the model's ability to fit the target data distribution, leading to improved perfor- mance -itto--distriuution (OOD) data. Specifically, by minimizing the perplexity P(y|æ; θ) of the answer y given the input which can be formulated as:
4.1. Perplexity Minimization for Test- Time Learning
min Plyla; θ) min e El logp(vals.))) (3) e
Perplexity (Bengio al., 2000) is widely used measure in language modeling that quantifies how well a model predicts sequence of tokens (Devlin et al., 2019; Brown et al., 2020). Given a sequence tokens {#1, 222 ..., xr ). the perplexity P is defined as the exponentiation average negative likellildoood of the predicted tokens:
This minimization process improves the model's perfor- mance the target data distribution. However, during the testing phase, we only access the user's input and not the ground truth output y. To address this limitation, hypothesize that minimizing the perplexity of the input denoted as min P(x; θ), may reduce the perplexity of the model's response y. The mathematical justification for this transformation is based on the assumption that the model parameters influence both P(y x.; θ) and P(r; θ) in a
the
P(1 xT]} - ΣT_
where log ①) is the conditional probability of predicting token I; given the previous tokens, parameter-

---

Test-Time Learning for Large Language Models
related manner, which can be described follows:
Assumption 1 (Autoregressive Property): The LLM gen- erates each token y, based on the input x and previously generated tokens V1:t-1: P(y (z, 31.e-1: O). The standard next-token prediction objective makes model predictions inherently conditional on previous context quality.
0.6
Origin LLM
Pull-Parmm LoRA
0.4
0.2
Assumption 2 (Shared Parameter Influence): LLM pa- rameters O influence both the input perplexity Pax; θ) and the conditional output perplexity P(y|ar; @). This assump- tion is valid across various LLM architectures, such as encoder-only and decoder-only models.
Googmaphy
Agriculture
Medicino
Financc
Figure 2. Comparison of prevent forgetting on DomainBench un- der Llama3. 1--B-l...--BBaa 1-8B-Instruct. This observation reveals that LoRA (Hu et al., 2022) prevents catastrophic forgetting more effectively than Full- Full-Param updates across DomainBench (see Supp. B).
Reducing Output Perplexity through Input Perplexity Minimization. Minimizing the perplexity to the input P(x; θ) is equivalent to maximizing the input generation probability P(æ; 6). We employ gradient-based theo- retical analysis to formalize the intuition that question- conditioned updates benefit answer predictions, based on a key assumption. Let O' = θ - 1, (- log P(æ; θ)) de- note the updated parameters after a single TTL step. Using a first-order Taylor expansion:
ing degrees of training (for ease of presentation, we show the normalized results here). As shown in Figure 1b, the relationship between input perplexity P(z; θ) and output perplexity Plylar θ) demonstrates a strong positive corre- lation across all four vertical domains. This indicates that reducing output perplexity is possible by minimizing imput perplexity in LLMs.
log Per (y(lx) (17") + Pe(yla)
(4)
n Ve log O)) Ve log (o(y)))
Cross gradient term
4.2. Sample Efficient Learning Strategy
where is the answer to the question x. Our core assumption
Minimizing input perplexity P(æ:; θ) can enhance the per- formance of L.MSs on target distribution data, as shown in Sec. 4.1. However, our intuition is that different test samples may produce varying effects during Test-Tine Time Learning. To investigate this, we conduct a preliminary study, leading to the following observation:
is that (Va, v Ve log P(x; 0)] Ve log (o(y(z) > O for question-answa answer pairs with strong semantic alignment. Under this condition, the cross-gradient term becomes non- negative, guaranteeing: log (y| 2 log (y(zz) for small 17 (We compute the gradient inner product using 400 batches (batch size- 50) of QA pairs from the Domain- Bench LLaMA3.1 1-BB. Results show 98. 75% of batch- samples satisfy the non-negativity condition, with average (V., V y +5.60).
Observation 2: high-pperllxity samples contribute more to LLM updates than low- uww-perplyxity ones. We select dif- ferent proportions of samples (the samples are pre-sorted according to their perplexity values P(x; Q)) for Test-Time Learning, and the resulting model is evaluated on all test samples. As shown in Figure 1c, we find that: 1) training the test samples with tigh-perplextty makes more contri- bution than low-perplexity ones, and 2) training on test samples with very low-perplexity may hurt performance. The possible reason is that low-perplexity samples are al- ready well-modeled by the pre-trained LLMs, offering little new information for further learning, which could lead to overfitting or a lack of generalization. In contrast, high- perplexity samples present more challenging data, driving greater adaptation during Test-Time Learning.
This form is consistent with the autoregressive property in Assumption 1. Naturally, based on the Shared Parameter Influence Assumption 2, minimizing (a:; θ) enhances the model's overall understanding and representation of r. This improved representation facilitates more accurate and confident next-token predictions, which is expected to reduce Ply|æ 9). To further investigate this, we conduct preliminary study, leading to the following observation:
Observation Trend of LLM's perplexity to the in- put P(x; θ) and perplexity to the output P(y|2; O) is the same. In the context of LLMs, it observed that the perplexity associated with the input P(:; θ) and the perplexity of the output Plylar; θ) exhibit similar trends. Specifically, we compute the trends of the perplexity of the input P(r; θ) and the perplexity of the output P(ylæ; lz; θ) on the four collected vertical domain datasets (see Supp. B) using Llama3.1 1----Instrue (Dubey et al., 2024) with vary-
Based Observation 2, we propose Sample Efficient Learning Strategy to actively select samples for backprop- agation, thereby enabling efficient Test-Time Learning. Specifically, we design an active sample selection score for each sample, denoted as S(x). The criterion is that a sample should be informative for Test- Tist-Time Learning, pro-
5

---

Test-Time Learning for Large Language Models
Table Comparison of experimental results the DomainBench and InstructionBench of the AdaptEval (see Supp. B). We mark the better scores in bold for better visualization and easier interpretation.
DomainBench
InstructionBench Dolly
Method
Geography
Agriculture
Medicinc
Finance
Alpaca-
Instruction Wild
Llama3.2
0.2395
0.0850
0.1411
0.2229
0.3564
0.3378
0.2562
Tent
0.1825
0.0150
0.1571
0.1093
0.0336
0.2105
0.0264
EATA
0.0064
0.0227
0.0259
0.0149
0.1410
0.0090
0.0122
COME
0,1000
0.1181
0.1542
0,1200
0.0437
0.2186
0.0697
TLM (Ours)
0,2893
0.1687
0.2308
0,2953
0,3883
0,3470
0,2824
Llama3- 3B-IInstruct
0.2450
0.0834
0.1265
0.2329
0,3752
0,3671
0.2608
Tent
0.0778
0.0067
0.0105
0,0372
0.2001
0,0036
0.0820
EATA
0,2081
0.0017
0.0127
0,1257
0.1397
0.1725
0,1088
COME
0,0048
0.0039
0.0301
0.0328
0.1424
0.0700
0.0240
TI.M (Ours)
0.3212
0.1319
0.2372
0.3242
0.4274
0.3785
0.2932
Ilamm2-11 13B-chat
0,2182
0.0840
0.1315
0,2382
0.3741
0,2892
0.2781
Tent
0.0320
0.0196
0.1131
0,0049
0.0955
0.0076
0.1108
EATA
0.2800
0.0771
0.1348
0.1155
0.0811
0.0513
0.1006
COME
0.1981
0,0380
0.1239
0,0172
0.0806
0.0000
0.0189
TLM (Ours)
0,2668
0.1013
0.2179
0,2760
0.3966
0.3007
0.2865
Qwen2.5-7B- 2ann2.5-7B-nnstrucc Instructt
0,2649
0.0981
0.1313
0,2739
0.4439
0,3121
0,2866
Tent
0,2362
0.1180
0.0524
0.1648
0.2132
0.1946
0.1710
EATA
0.2109
0.1203
0.1334
0.2846
0.0000
0.2056
0.1710
COME
0,2306
0,1180
0.0463
0.1780
0.3781
0.2182
0.1710
TLM (Ours)
0.3081
0.1652
0.2394
0.3311
0.4608
0.3177
0.3482
viding enough information to drive the model's learning process, referred to as an informative sample. By setting (xx) 0 for uninformative samples, we can reduce un- necessary backpropagation computations during Test-Time Learning, thereby improving the overall efficiency. Relying on the sample score Szz), we use perplexity loss for model training. Then, the sample-efficine perplexity minimization is to minimize the following objective:
Learning on DomainBench (see Supp. B) using both Full- Param and Low-Rank Adaptation (LoRA) (Hu al., 2022) updates, and evaluate the LLM'' performance on GSM8K (Cobbe et al., 2021). From Figure 2, we observe that LoRA, compared to Full-Param updates, better preserves the model's originally learned general knowledge, thereby demonstrating significant regularization effect. This is likely due to LoRA's ability to fine-tune only small subset of model parameters, which effectively reduces the risk of overfitting and catastrophic forgetting.
min (zz)
(5)
To obtain the active sample selection score S(x:), we pro- pose crpplextty-based weighting scheme to accurately identify reliable samples and emphasize their contribution Fest-Time Learning. Formally, the active sample selec- tion score S(x) can be calculated follows:
Based on Observation Learning, where the optimization objective is Eqn. 5 is modified accordingly follows:
adopt the LoRA for Test-Time
min S(n)P()))(;; = min S(z)P(r; A0), (7)
26
(6) where (-) an indicator function, À and Po are pre- defined threshold. The above weighting function excludes low powpperpyyxtty samples from Test-Time Learning and as- signs higher weights to high-perplexity test samples, en- abling them to contribute more significantly to model up- dates. It is important to note that evaluating S(x) does not involve any gradient backpropagation.
5(x) = A flog P(a:0) log Pol. [(P(=:))>P..)x)
where AO = BA is zero at the beginning of training, with A using random Gaussian initialization and set to zero, and we update only AO during the Test-Time Learning.
5. Experiments
5.1. Experimental Settings
Datasets. To evaluate the effectiveness of our TLM, construct comprehensive benchmark named AdaptEval, designed to cover diverse tasks and domains. AdaptEval consists three categories datasets. 1) DomainBench includes four vertical domain knowledge datasets: Geogra- phy, Agriculture, Medicine, and Finance, and is designed
4.3. Modulating Parameters for Test-Time Learning
Observation3: Low-Rank Adaptation prevents catas- trophic forgetting more effectively than Full-Param up- dates during test-time learning. We conduct Test-Time
6

---

Test-Time Learning for Large Language Models
Table Comparison experimental results on the ReasoningRench.
Table 4. Experimental results for the component of our proposed
method on the DomainBench of the AdaptEval. The SEL means "Sample Efficient Learning Strategy" and the (-) indicates relative improvement over the result in the previous column.
ReasoningBench MetaMath
Method
GSM8K
Logiqa
Llama3.2-33 1a.d332----nntruuee
0.7756
0.7976
0,4194
Tent
0.7726
0.7412
0,4012
Version
Llama3-8B
Ours (w/o SEL)
Ours
EATA
0.0032
0.0310
0.0284
Gengraphy
0.2450
0.31 0.3190k330.25)
0.3212(ca.79)
COME
0.7710
0.7308
0.4196
Agriculture
0.0834
0.1255ce30.31)
0.1319(+519)
TLM (Ours)
0.9096
0.8818
0.4572
Medicine
0.1265
0.2326x+83399)
0.2372(2208)
Bamma--8iB-Insttuue
0.7610
0.6912
0,4550
Finance
0.2329
0.3222
0.3242(v0.6%)
Tent
0.7578
0.6550
0,4378
#Backwards
5000
4772(4689)
EATA
0.0250
0.5454
0,2192
COME
0.7479
0.6460
0.2180
TLM (Ours)
0.8074
0.7006
0.4868
(ama2-113-chaa
0.3458
0.2498
0.3992
Tent
0.2706
0.0040
0.2566
rule, with a batch size of and the learning rate of 5e-5/ 5e-5/ 1e-6 6 for DomainBench/ InstructionBench/ Reason- ingBench. The À and Po in Eqn. 6 are set to 0.10 and ea. To improve the stability of outputs produced by LLMs, we apply greedy decoding with temperature of 0 across all experiments. More details in Supp. C.2. The source code is available at htt.ggitt...ohuunnseeeeeeeeeeeee
EATA
0,3392
0.0572
0.2606
COME
0.3272
0.2646
0,2462
TLM (Ours) 0.3508
0.2576
0.4124
Qwen2.5-77B 1wt.S----B-Innteucc
0.8378
0.7430
0,5952
Tent
0.8455
0.7412
0.5934
EATA
0.7098
0.0070
0,2172
COME
0.8556
0.7559
0,5908
TLM (Ours)
0.8424
0.7560
0.6046
5.2. Comparison Experiments
We compare our proposed TLM, the original LLM, Tent, EATA, and COME to demonstrate the superior performance of our method. We conduct experiments on different types of datasets, including DomainBench, InstructionBenchh, and ReasoningBench, summarized in Table 2 and 3. More detailed results can be found in Supp. D.
evaluate the I.LM adaptability to specialized fields. 2) In- structionBench contains three
instruction- following datasets: Alpaca-GPT4, Dolly, and Instruction- Wild, and focuses the LLM adaptability to instruction- based tasks. 3) ReasoningBench comprises three reasoning capability datasets: GSM8K, MetaMath, and Logiga, and aims to assess the LLM logical reasoning and problem- solving abilities. These datasets collectively form a diverse and challenging evaluation suite, designed thoroughly assess the effectiveness of TLM in adapting LLMs tasks requiring vertical knowledge, instruction- -follooinnnollowiingg capabil- ities, and logical reasoning under distribution shifts. More details can be found in Supp. B.
Our proposed TLM is consistently better than the orig- inal LLMs. From Table and 3, our method consistently outperforms the original LLMSs across all types of datasets and different LLM architectures. For instance, on the four datasets of DomainBench, the proposed TLMM achieves least a 20.00% improvement over the original LLMs. Specif- ically. on the Geography dataset, our proposed TLM im- proves performance by a relative 20.79% (0.2395 + 0. 0.2893) compared to Llama3.2-2 2----B--Innsruct.
Metries. We use Rouge-Lsum (R-Lsum) (Lin, 2004) as the evaluation metric for DomcinBench and InstructionBench, while Exact Match (EM) (Chang et al., 2024) is used for ReasoningBench. More metrics can be found in Supp. C.I.
Superior performance on Domain Knowledge Adapta- tion. To evaluate the effectiveness of proposed TLM in adapting to vertical domain knowledge, ments on DomainBench, which includes four datasets. From Table 2, the results demonstrate that the proposed TLM out- performs the original L.LMs, Tent, and EATA, achieving significant performance improvements. For example, in test-time updating of model parameters on Qwen2.5-7B- Instruct, the proposed method yields relatively 37.32% (0.1203 - 0.1652) improvement the Agriculture dataset compared to the EATA.
conduct experi-
LLMs and Baseline. We use diverse range of LLMs of varying sizes and types, including Llama3.2- 3--.3..2---B-nttttee Instruct, LJama3-8B-Insttee (Dubey et al., 2024), Llama2-13BB 13B-Chat (Touvron et al., 2023a), and Qwwe2.55-7BIInstruce (Yang et al., 2024). We evaluate our TLM against the baseline methods, Tent (Wang et al., 2021), EATA (Niu al., 2022a), and COME (Zhang et al., 2025). They are state-of-thed -heth-rrt TTA methods that update model parameters using unlabeled data. We adapt Tent, EATA, and COME to the offline setting for a fair comparison. The implementation details can be found in the Supp. C.
Superior performance on instruction-based task. As shown in Table 2, our proposed TLM achieves substan- tial improvements over the original LLMs and Tent across instruction-basee datasets. For instance, on the A Alpaca-
Implementation Details. We use A as the update
7

---

Test- Test-Time Learning for Large Language Models
Table 5. Experimental results of our proposed method in the Online setting. Geo., Agri., Med., and Fin. represent the Geography, Agriculture, Medicine, and Finance, respectively. LLM refers to Llama3-8B-Instruct NF4 is 4-bit NormalFloat.
0.30
ROUGE-LSUM
0.25
Geogmahy
Agriculture
0.20
Medicine
DemainBench
Finance
Method
Geo.
Agri.
Med.
Fin.
#Backwards
0.15
LLM
0.2450
0.0834
0.1265 0.2329
0.10
Tent (Online)
0,0804
0.0112
0,0142
0.0489
5000
EATA (Online)
0.1008
0.0186
0.0202
0.0815 4943(1119
e²
6
Ours (Online)
0.2787
0.1579
0.1340
0.2455
1514.407%)
LLM (NF4)
0.2439
0.0859
0.1237
0.2325
Figure 3. Effects of different perplexity margins Po in Eqn. 6.
Ours (NF4)
0.3069
0.1533
0.2306
0.3193
4783(-43%)
GPT4 dataset, our proposed TLM improves the perfor- mance of Llama3.2-88 22-B-Innstuue by 13.91% (0.3752 0.4274), showing relative improvement of about 113.60% (0.2001 + 0.4274) compared to Tent, demonstrating its effective adaptation to general instruction-follovii tasks.
shown in Table 4, by prioritizing the most informative and relevant test samples for backpropagation, this strategy not only further enhances LLM performance but also reduces unnecessary computational overhead. Specifically, using the Sample Efficient Learning Strategy results in relative performance improvement of approximately 2.0% on the target test data, while reducing the training data by about 5.0%. For instance, the relative performance improvement on the Agriculture dataset is 5.1%
Superior performance on logical reasoning task. As shown in Table 3, our proposed TLM significantly outper- forms the original LLMs and Tent on all reasoning datasets in ReasoningBench. For instance, on the GSM8K dataset, our proposed TLM improves the performance of Llama3- 8B-Instruct by 6. 10%, highlighting its ability to enhance logical reasoning under complex arithmetic and problem- solving tasks. These results confirm that our method not only adapts effectively to distributional shifts but also en- hances the reasoning capabilities of LLMSs during test time.
Effects of the Po in Eqn. 6. The threshold Po in Eqn. 6 plays a crucial role in controlling the threshold for sample selection during Test-Time Learning. To explore the optimal threshold for Po. we conduct experiments with values of Po set to (c² e² eA, ,,e", e²). As shown in Figure 3, when Po = = our method achieves the best performance on three datasets, namely Geography, Medicine, and Finance, while also showing near-optimal performance on the Agriculture dataset. Therefore, we select Po = e for all experiments. When Po set too high or too low, it negatively affects performance. Specifically, a value that is too high restricts the number of perpleetyyy samples selected, limiting the model's ability to adapt to new and complex data. On the other hand, value that is too low includes too many low-perplexity samples, which do not contribute effectively to adaptation and could lead to inefficiencies and overfitting.
5.3. Ablation Studies
Effectiveness of Input Perplexity Minimization. To eval- uate the effectiveness of input perplexity minimization, we conduct experiments comparing the performance of Llama3- 8B- SB-Instruct and our method without the Sample Efficient Learning Strategy (Ours w/o SEL). As shown in Table 4, input perplexity minimization significantly improves the performance of LLMs across all datasets compared to the original Llama3-8B -llama--B-Instruuue I--nttutt demonstrating that minimizing input perplexity effectively enhances the L.M''s ability to adapt target domains. Specifically, by minimizing input perplexity during Test-Time to update the LLM parameters, we achieve relative performance improvement of over 30% compared to the original Lamaa-88-Inssruus model. No- tably, on the Medicine dataset, the improvement reaches 83.9%. The effectiveness of input perplexity minimization P(x; θ) lies in its ability to enhance the LLM's understand- ing and representation of the input, which helps improve the model's 's adaptation to the target domain data.
5.4. More Discussions
Online Test-Time Experiments. To further assess the performance of our TLM, we conduct experiments in the online Test-Time Learning setting. The online setting is similar to Test- Time- Adaptation (Wang et al., 2021; Niu et al., 2022a), where the model processes the input to gener- ate an output while simultaneously updating its parameters. Notably, the model parameters are updated only once ev- ery 100 test samples. From Table 5, the proposed method also achieves significant performance improvements over Llama3-88 Baauaa--B-Imnan Instruct across different domain datasets in the online setting. Additionally, our proposed method reduces
Effectiveness of Sample Efficient Learning Strategy in Eqn. 5. The Sample Efficient Learning Strategy improves the efficiency of TTL by actively selecting high-perplexity samples that contribute more to the LLLM's adaptation. As
8

---

Test-Time Learning for Large Language Models
the number of backward by 69.7% (5000- + 1514) in the online setting. This is because, as the L.M is updated, some samples progressively become easier for the model, and are thus excluded from TTL in Eqn. (6)
specifically highlighted here.
References
Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. I.., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
Experiments on Quantized LLM. To evaluate the perfor- mance of our method on quantized LLMs, we conduct exper- iments on 4-bit quantized version of -B-a--------ntcctt following the settings of QLoRA (Dettmers et al., 2024). From Table 5, method also demonstrates strong perfor- mance on target domain datasets when applied to quantized LLMs. Specifically, the proposed method improves at least 25.0% cover Mama3-8B-Instruus (NF4) on four datasets on DomainBench, highlighting the broad applicability of our TTL scheme.
Akyürek, E., Damani, M., Qiu, L., Guo, H., Kim, Y., and An- dreas, J. The surprising effectiveness of test-time training for abstract reasoning. arXiv preprint arXiv2241..0727, 2024.
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:230S. 10403, 2023.
6. Conclusion
In this paper, we propose novel Test-Time Learning (TTL) method for Large Language Models (LLMs), named TLM, to address the challenges posed by dynamic and diverse data distributions in real-world environments. By leverag- ing only unlabeled test data, TLM efficiently adapts LLMs and improves their robustness in target domains. Specifi- cally, through observation and theoretical analysis, we argue that reducing output perplexity can be achieved by minimiz- ing the imput perplexity of unlabeled test data. Based on this insight, we adopt input perplexity minimization as the op- timization objective for test-time LLM updates. Moreover, we find that gegh-eerpeextty test samples play a more crucial role in model updates than low-perplexity samples. This insight motivates the development of our Sample Efficient Learning Strategy, which actively selects and emphasizes high-perplexity test samples for backpropagation. optimiz- ing the learning process. Lastly, we reveal that Low-Rank Adaptation is more effective than full parameter updates in mitigating catastrophic forgetting, and we utilize for parameter updates during TTL, enabling lightweight model adaptation while effectively preserving prior knowledge.
Asai, A.. Wu, Z. Wang. Y., Sil, A., and Hajishirzi, H. Self- rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth International Conference on Learning Representations, 2024.
Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprint ar Xivv:2309. 16609, 2023.
Bartler, A., Bühler, A., Wiewel, F,, Döbler, M., and Yang, B. M13: Meta test-time training for self-supervised test- time adaption. In International Conference on Artificial Intelligence and Statistics, pp. 3080-3090. PMLR, 2022.
Bella, G., Helm, P., Koch, G., and Giunchiglia, F. Tackling language modelling bias in support of linguistic diversity. In The 2024 ACM Conference Fairness, Accountability, and Transparency, pp. 562-572, 2024.
Bengio, Y., Ducharme, R., and Vincent, P. A neural proba- bilistic language model. Advances in neural information processing systems, 13, 2000.
Acknowledgements
Boudiaf. M., Mueller, R., Ben Ayed, I., and Bertinetto, L. Parameter-free online test-time adaptation. In Proceed- ings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 8344-8344 8353, 2022.
This work was partially supported by the Joint Funds of the National Natural Science Foundation of China (Grant No.U24A20327), Key-Area Research and Development Pro- gram of Guangdong Province (2018B010100 Science and Technology Innovation Fund, and the Young Scholar Project of Pazhou Lab (No. 2000202
TCL
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877-1901, 2020.
Impact Statement
This paper presents work whose goal advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be
Chang, Y., Wang, X., Wang, J., Wu, Y., Yang, L., Zhu, K., Chen, H., Yi, X., Wang, C., Wang, Y., et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 15(3):1-45, 2024.

---

Test-Time Learning for Large Language Models
Chen, G., Niu, S., Chen, D., Zhang, S., Li, C., Li, Y., and Tan, M. Cross-device collaborative test-time adaptation. In The Thirty-eighth Annual Conference on Neural Infor- mation Processing Systems, 2024a.
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle,
A., Letman, A., Mathur, A., Schelten, A., Yang. A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407. 2024.
Chen, M., Tworek. J., Jun, H.. Yuan, Q. Pinto, O.D.O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint rrXiv:2107.03332 2021.
Fan, W., Ding, Y., Ning, L., Wang, S., Li, H., Yin, D., Chua, T.-S., and Li, Q. A survey on rag meeting llms: Towards retrieval -aumeetl- large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 6491-6501, 2024.
Chen, Y., Niu, S., Wang, Y. Xu, S., Song. H.,, and Tan, M. Towards robust and efficient cloud-edge elastic model adaptation via selective entropy distillation. In The Twelfth International Conference Learning Represen- tations, 2024b.
Fleuret, F. et al. Test time adaptation through perturbation robustness. In NeurlPS 202/ Workshop on Distribution Shifts: Connecting Methods and Applications, 2021.
Gandelsman, Y., Sun, Y., Chen, X., and Efros, A. Test-time training with masked autoencoders. Advances in Neural Information Processing Systems, 35:29374-29338 2022.
Chen, Y. Qian, S., Tang, H., Lai, X., Liu, Z., Han, S., and Jia, J. Longlora: Efficient fine-tuning of contecontext large language models. In The Twelfth International Conference on Learning Representations, 2024c.
Gao, J., Zhang, J., Liu, X., Darrell, T., Shelhamer, E., and Wang, D. Back to the source: ceffusiin-drivenr adaptation to test-time corruption. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 17786-11796, 2023.
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung. H. W., Sutton, C., Gehrmann, S., et al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1-113, 2023.
Gu, Y., Tinn, R., Cheng, H.,, Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., and Poon, H. Domain-specific language model pretraining for biomedical natural lan- guage processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23, 2021.
Chung. H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fe- dus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-fiettune language models. Journal of Machine Learning Research, 257)) 2024.
Hardt, M. and Sun, Y. Test-time training on nearest neigh- bors for large language models. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint wwX::x2110.111000 2021.
He, P., Liu, X., Gao, J., and Chen, W. Deberta: Decoding- enhanced bert with disentangled attention. In Interna- tional Conference on Learning Representations, 2021.
Dettmers, T., Pagnoni, A., Holtzman, A., and Zeulemoyer, L. Qlora: Efficient finetuning of quantized lims. Advances in Neural Information Processing Systems, 36, 2024.
Hong, J., Lyu, L., Zhou, J., and Spranger, M. Mecta: Memory-econone continual test best-time model adaptation. In 2023 International Conference on Learning Represen- tations, 2023.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding, 2019. URL https: htg/abs//10 1et../111..00 04805.
Hu, F. J., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., al. Lora: Low-rank adaptation of large language models. In International Conference on Learn- ing Representatiinss 2022.
Du, Z., Qian, Y., Liu, X., Ding, M., Qiu, J., Yang, Z., and Tang, J. GLM: General language model pretrain- ing with autoregressive blank infilling. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.), Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 320-335, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/w1/202. acl-long.26. URL https: htt///aaaannnlogoooee org/ 2022. -lng 26/.
Hu, J., Wang, Y., Zhang, S., Zhou, K., Chen, G., Hu, Y., Xiao, B., and Tan, M. Dynamic ensemble reasoning for Ilm experts. Proceedings of the Fourth-Fourth Interna- tional Joint Conference on Artificial Intelligence, 2025a.
Hu, J., Zhang, W., Wang, Y., Hu, Y., Xiao, B., Tan, M., and Du, Q. Dynamic compressing prompts for effi- cient inference of large language models. arXiv preprint arXiv:2504. 1004, 2025b.
10

---

Test-Time Learning for Large Language Models
Hübotter, J., Bongni, S., Hakimi, I., and Krause, A. Effi- ciently learning test-time: Active fine-tuning of lmms. In NeurlPS 2024 Workshop on Mathematics of Modern Machine Learning, 2024.
Twelfth International Conference on Learning Represen- tations, 2024. URL https: htwp:////ppeeeeeeeee/mee net/ forum?ide
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mo- hamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. Bart: Denoising Sequence-to-sequeee
Iwasawa, Y. and Matsuo, Y. Test-time classifier adjustment module for model-agnostic domain generalization. Ad- vances in Neural Information Processing Systems, 34: 2427-2440, 2021.
pre- pre-training for natural language generation, translation, and comprehen- sion. In Proceedings of the 58th Annual Meeting of the As- sociation for Computational Linguistics, pp. 7871-7880, 2020a.
Jiang. Z., Xu, F., Gao, L., Sun, Z., Liu, Q., Dwivedi- Yu, J., Yang, Y., Callan, J., and Neubig, G. Active retrieval augmented generation. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7969-7992, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18653/112023. emnlp- 491pp-main.495. URL https: pesss//aacaanne haclaaaaaanooooyy... org/2023. emnlp main. 495/.
Lewis, P.. Perez, E., Piktus, A., Petroni. F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t,, Rocktäschel, T., et al. Retrieval-augmented generation for knowledge- intensive nlp tasks. Advances in Neural Information Pro- cessing Systems, 33:9459-94777 9474,, 2020b.
Li, X., Nic, E., and Liang, S. From classification to gen- eration: Insights into crosslingual retrieval augmented ICL. In NeurIPS 2023 Workshop on Instruction Tun- ing and Instruction Following, 2023. URL https: / //pppnneview..... net
Jiang, Z., Ma, X., and Chen, W. Longrag: Enhancing retrieval-augnee augmented generation with long-contex: Ilms. arXiv preprint arXiv:2406. 25:::239..//1,9, 2024.
Liang, J., He, R., and Tan, T. A comprehensive survey on test-time adaptation under distribution shifts. Interna- tional Journal of Computer Vision, pp. 1-34, 2024.
Jumelet, J. and Zuidema, W. Transparency at the source: Evaluating and interpreting language models with access to the true distribution. In Findings of the Association for Computational Linguisties: EMNLP 2023, pp. 4354 4369, 2023.
Lin, C.-Y. Rouge: package for automatic evaluation of summaries. In Text summarization branches out, pp. 74-81, 2004.
Karpukhin, V. Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Web- ber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769- 6781, Online, November 2020. Association for Computa- tional Linguistics. doi: 10. 00.18653/11/0222mmmpp- 550. URL https: / //a/ccaannn /ww.aaatllloooco.cccocc org/2000. emnlp-ma .tt.mmain.5555.. 550/.
Lin, X. V., Chen, X., Chen, M., Shi, W., Lomeli, M., James, R., Rodriguez, P., Kahn, J., Szilvasy, G., Lewis, M.. Zettlemoyer, L., and tau Yih, W. RA-DIT: Retrieval- augmented dual instruction tuning. In The Twelfth In- ternational Conference on Learning Representations, 2024. URL hettps://openreviie net forum? i id=220Tbutug9.
Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11666 364, 2019.
Kim, G., Kim, S.. Jeon, B., Park, J., and Kang, J. Tree of clarifications: Answering ambiguous questions with retrieeel-augmeentd large language models. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL https: opwnreviee..eeeeeeeeeeeeeeeee..r net/ forum? 1
Liu, Y., Kothari, P., Van Delft, B., Bellet-Gurlet, B., Mordan, T., and Alahi, A. Tit++; When does self-supervised test-time training fail or thrive? Advances in Neural Information Processing Systems, 34:2 14:21808-21822 2021.
Mirza, M. J., Soneira, P. Lin, W., Kozinski, M., Possegger, H., and Bischof, H. Actmad: Activation matching to align distributions for test-time-training. In Proceedings of the IEEF/VVF Conference on Computer Vision and Pattern Recognition, pp. 24152-24161, 2023.
Le Scao, T., Fan, A., Akiki, C., Pavlick, E., Ilié, S., Hesslow, D., Castagné, R., Luccioni, A. S., Yvon, F., Gallé, M., al. Bloom: A 176b-parameter open-access multilingual language model. 2023.
Nado, Z., Padhy, S., Sculley, D., D' Amour, A., Lakshmi- narayanan, B., and Snoek, J. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006. 10963, 2020.
Lee, J., Jung, D., Lee, S., Park, J., Shin, J., Hwang, U., and Yoon, S. Entropy is not enough for rest-time adapta- tion: From the perspective of disentangled factors. In The

---

Test-Time Learning for Large Language Models
Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P., and Tan, M. Efficient test-time model adaptation with- out forgetting. In International conference machine learning, pp.
Reimers, N. and Gurevych, I. Sentence-bert: Sentence em- beddings using siamese bert-networks. In Conference on Empirical Methods in Natural Language Processing, 2019. URL https: /ppi. semanticscho org/ C///oorruuappp 201
PMLR, 2022a.
201 64 6309.
Niu, S., Wu, J., Zhang, Y., Xu, G., Li, H., Zhao, P., Huang, J., Wang, Y., and Tan, M. Boost test-time performance with closed-loop inference. arXiv preprint arXiv:2203. 10853, 2022b.
Ren, Y., Cao, Y., Guo, P., Fang, F., Ma, W. and Lin, Z. Retrieve-and -os--aanddd---ee- Document-level event argument extraction via hybrid retrieval augmentation. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 293-306, Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.18653/11/2023. acl-long.17. URL https::/t//// wtt.tttsss//aaattttttotoooo http://aaaantttooooy..oo org/ 2023 acl long. 17/.
Niu, S., Wu, J., Zhang, Y., Wen, Z., Chen, Y., Zhao, P., and Tan, M. Towards stable test-time adaptation in dynamic wild world. In The Eleventh International Conference on Learning Representations, 2023.
Niu, S., Miao, C.,, Chen, G., Wu, P., and Zhao, P. Test-time model adaptation with only forward passes. In Forty- first International Conference on Machine Learning, 2024. URL https:t////// http///pppeeeeeeeeeeeeeeeeeeeeee htw.//ee//ete. net/forum? forum? id=gz1Vx1v9ik,
Schneider, S., Rusak, E., Eck, L.,, Bringmann, O., Bren- del, W. and Bethge, M. Improving robustness against common corruptions by covariate shift adaptation. Ad- vances in neural information processing systems, 33: 11539-11551, 2020.
Oh, Y.. Lee, J., Choi, J., Jung, D., Hwang, U., and Yoon, S. Efficient diffusion-driven corruption editor for test-time adaptation. In European Conference on Computer Vision, pp. 184-201 Springer, 2025.
Shao, Z., Yu, Z., Wang, M., and Yu, J. Prompting large language models with answer heuristics for knowledge- based visual question answering. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp. 14974-14983, 2023.
Papinnni, K., Roukos, S., Ward, T., and Zhu, W.-J Bleu: a method for automatic evaluation of machine transla- tion. Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311-318, 2002.
Shu, M., Nie, W., Huang, D.-A., Yu, Z., Goldstein, T., Anandkumar, A., and Xiao, C. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35: 14274-14289, 2022.
Peng, B., Galley, M., He, P., Cheng, H., Xie, Y., Hu, Y., Huang, Q., Liden, L.. Yu, Z., Chen, W., al. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint 2r3:2 128/3, 2023.
Song, J., Lee, J., Kweon, I. S., and Choi, S. Ecotta: Memory- efficient continual test-time adaptation via self-distilled regularization. In Proceedings of the IEEE/CVE Confer- ence on Computer Vision and Pattern Recognition, pp. 1 -1920-11229, 2023.
Qian, H., Zhang, P., Liu, Z., Mao, K., and Dou, Z. Mem- orag: Moving towards next-gen rag via memory innsiieed knowledge discovery. arXiv preprint arXiv:2409.05591, 2024.
Sun, Y., Wang. X., Liu, Z., Miller, J., Efros, A., and Hardt, M. Test-time training with self-supervision for generaliza- tion under distribution shifts. In International conference on machine learning, pp. 9229-9222 PMLR, 2020.
Radford, A. Improving language understanding by genera- tive pre-training. 2018.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 (140):1-67, 2020.
Tay, Y., Dehghani, M., Tran, V. Q., Garcia, X., Wei, J., Wang, X., Chung, H. W., Bahri, D.,, Schuster, T., Zheng, S., Zhou, D., Houlsby, N., and Metzler, D. UL2: Uni- fying language learning paradigms. In The Eleventh International Conference Learning Representations, 2023. URL https: /wwp//weee//weeeeeoe//e/o.oo..wa net forum? id=6ruVLB727MG
Ram, O., Levine, Y., Dalmedigos, I., Muhlgay, D.,, Shashua, A., Leyton-Brown, K., and Shoham, Y. In-context retrieval- retrieval-augmented language models. Transactions of the Association for Computational Linguistics, 11:1316- 1331, 2023. doi: 10.1162/ual.a.0066) URL https: //aclanthol 1 loy. org.oggg tacl-1.71..
Thirunavukarasu, J., Ting, D. S. Elangovan, K., Gutier- rez, L., Tan, T. F., and Ting, D. S. W. Large language models in medicine. Nature medicine, 29(8):1930-1948 2023.
12

---

Test-Time Learning for Large Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation lan- guage models. arXiv preprint arXiv:2302.13977 2023a.
openreview. net http://wwnnee//foomiiddiedee http://fummmmiiidsssssssss Sur- vey Certification.
Wei, J., Wang. X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V. Zhou, et, et. Chain-of of-thhuugtttt prompting elicits reasoning in large language models. Advances in neural information processing systems, 15:24824-24831 2022b.
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine- tuned chat models. arXiv preprint 0rX0v22307.09288, 2023b.
Yang, A., Yang. B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., et al. Qwen2. technical report. arXiv preprint 1511v:4412.55115, 2024.
Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. Interleaving retrieval with chain- saain-oof-hhone -h-houhf reasoning for inowledge-intensive multi-step questions. In Rogers, A., Boyd- Boyd-Graber, J., and Okazaki, N. (eds.), Proceed- ings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 10014-10033 Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 10.11833/11/223.. acl-Iong.5577 httphhttss//claatttteee 2023. acl-allllaa long. 557/.
Yi, C., Chen, H., Zhang, Y., Xu, Y., Zhou, Y., and Cui, L. From question to exploration: Can classic test-time adaptation strategies be effectively applied in semantic segmentation? In Proceedings of the 32nd ACM Inter- national Conference Multimedia, pp. 10085-10094, 2024.
org/
Zhang, M., Levine, S., and Finn, C. Memo: Test time
robustness via adaptation and augmentation. Advances in neural information processing systems, 35:38629-38644 2022a.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, E., and Polosukhin, I. At- tention is all you need. Advances in neural information processing systems, 30, 2017.
Zhang, Q., Bian, Y., Kong. X., Zhao, P., and Zhang. C. Come: Test-time adaption by conservatively minimizing entropy. In The Thirteenth International Conference on Learning Representations, 2025.
Wang, D., Shelhamer, F.,, Liu, S., Olshausen, B., and Darrell, T. Tent: Fully test-time adaptation by entropy minimiza- tion. International Conference on Learning Representa- tions, 2021.
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., al. Opt: Open pre-trained transformer language models. arXiv preprint 2XXi:2205.00166 2022b.
Wang, Q. Hu, J., Li, Z., Wang, Y., Hu, Y., Tan, M., et al. Generating long-form story using dynamic hierarchical outlining with memory-enhancement The 2025 Annual Conference of the Nations of the Americas Chapter of the ACI., 2025a.
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:/904.000 1904.0965, 2019.
Wang, Y., Li, P., Sun, M., and Liu, Self-knowledge guided retrieval augmentation for large language mod- Empirical Methods
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z.,, et al. A survey of large language models. arXiv preprint arXiv:2303.1888 2023.
els. In The 2023 Conference in Natural Language Processing, 2023. URL. https: / //operreview net. forum? MddioddMM
Zhong, W., Guo, L., Gao, Q., Ye, H., and Wang, Y. Memo rybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, volume 38, pp. 19724-19731, 2024.
Wang, Y.. Hu, J., Huang. Z., Lin, K., Zhang, Z., Chen, P., Hu, Y., Wang, Q., Yu, Z., Sun, B., Xing, X., Zheng, Q. and Tan, M. Enhancing user-oriented proactivity in open-domain dialogues with critic guidance. Proceedings of the Thirth-Fourth International Joint Conference on Artificial Intelligence, 2025b.
Zhu, Y., Kiros, R., Zemel, R., Salakhutdinov, R., Urta- sun, R., Torralba, A., and Fidler, S. Aligning books and movies: Towards story-like visual explanations by watch- ing movies and reading books. In Proceedings of the IEEE international conference on computer vision, pp. 19 19-27,2015,
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Met- zler, D.,. Chi, E. H., Hashimoto. T., Vinyals, O., Liang, P., Dean, J., and Fedus, W. Emergent abilities of large language models. Transactions on Machine Learning Research, 2022a. ISSN 2835-8856.
URL https://
13

---

Supplementary Materials for "Test-Time Learning for Large Language Models"
In the Supplementary, we provide descriptions of more related works, details, and experimental results of the proposed TLM. We organize the supplementary into the following sections.
In Section A, we provide descriptions of related works regarding Large Language Models, Retrieval-Augmente Generation, and Test-Time Adaptation.
In Section B, we present a more detailed version our AdaptEval Benchmark.
In Section C, we present a more detailed implementation of our experiments.
In Section D, we report more results of our experiments.
In Section E, we provide the discussion of TLM and future directions.
A. More Related Work
A.1. Large Language Models
The rapid progress in natural language processing (NLP) has been marked by the emergence of large language models (LLMss, which have fundamentally transformed the landscape of artificial intelligence. These models, rooted in the Transformer architecture ( (Vaswani et al., 2017), leverage extensive pre-training capabilities. They have shown impressive performance across a wide range of tasks (Wei et al., 2022a; Hu et al., 2025b), such as high-quality question answering (Shao et al., 2023; Peng et al., 2023), coding (Chen et al., 2021), and intermediate reasoning (Wei al., 2022b). The unprecedented success of LLMs has spurred significant discussions regarding their application for achieving artificial general intelligence (AGI) (Zhao et al., 2023). Based their architectural design, existing I.MSs can be categorized into three major classes: encoder-only models, decoder-only models, and encoder-decoder models.
massive text corpora to acqire remarkable
Encoder-only models. The encoder-only models primarily employ the Transformer encoder to encode input sequences into rich contextual representations. They are particularly effective in natural language understanding (NLU) tasks, where the focus lies in extracting semantic meaning from text. One notable example is BERT (Devlin et al., 2019), which uses bidirectional encoding to capture context from both preceding and succeeding tokens. Pre-trained on extensive datasets like BooksCorpus (Zhu al., 2015) (800M words) and English Wikipedia (2,500M words), BERT set new benchmarks on datasets such as GLUE and MultiNLI. Subsequent iterations, including RoBERTa (Liu, 2019) and DeBERTa (He et al., 2021), introduced architectural refinements and improved pre-training strategies, further enhancing performance. Despite their strengths in understanding tasks, encoder-only models are inherently unsuited for tasks that require sequence generation, such as translation or text completion.
Decoder-only models. This kind of models, in contrast, rely solely on the Transformer decoder and are designed to generate text in an auto-regressive manner, where each token is generated sequentially, conditioned on previously generated tokens. Obviously These models excel in natural language generation (NLG) tasks, such as summarization, content creation and QA. The Generative Pre-trained Transformer (GPT) series (Radford, 2018; Brown et al., 2020; Achiam al., 2023) developed by OpenAl examines this class, with GPT-3 being landmark model that features 175 billion parameters. Trained diverse corpus spanning Common Crawl (Raffel et al., 2020), WebTex12, Books 1, Books 2, and Wikipedia datasets. GPT-3 has demonstrated extraordinary few-shot and zero-shot learning capabilities on many language tasks. In addition to GPT series, many decoder-only models have been developed, such as OPT, LLaMA, Llama2, Llama3 from Meta (Zhang et al., 2022b: Touvron et al., 2023a;b; Dubey et al., 2024), PaLM, PaLM2 from Google (Chowdhery et al., 2023; Anil et al., 2023), BLOOM from BigScience (Le Scao et al., 2023), and Qwen series from Alibaba (Bai et al., 2023; Yang et al., 2024).
14

---

Test-Time Learning for Large Language Models
However, decoder-only models, while excelling in generative tasks, often lack the nuanced comprehension abilities required for deep understanding of long and complex input contexts.
Encoder-decoder models. Encoder-decoder models incorporate both the encoder and decoder components of the Trans- former, combining the strengths of the two structures to handle tasks that require imput understanding and sequence generation. Prominent examples of encoder-decoder models include GLM from Tsinghua University (Du et al., 2022), T5, FLAN-TS, and UL2 from Google (Raffel et al., 2020; Chung et al., 2024; Tay et al., 2023), as well as BART from Meta (Lewis et al., 2020a). For instance, GLM adopts an autoregressive blank-infillinn objective to effectively address three core challenges in NLP: natural language understanding (NLU), unconditional text generation, and conditional generation. With maximum capacity of 130 billion parameters, it is pre-trained on datasets such as BookCorpus (Tay et al., 2023) and Wikipedia. GLM surpasses BERT on the SuperGLUE benchmark by 4.6%-5.0% and demonstrates superior performance compared to FLAN-TS on both NLU and generation tasks using fewer parameters and training data.
A.2. Retrieval-Augmented Generation
As one of the most representative techniques in the field of generative AI, Retrieval-Augmennté Generation (RAG) aims to enhance the quality of the generated text content with retrieved information. It achieves this by integrating two critical components: (i) a retrieval mechanism that accesses relevant documents or information from external knowledge sources, and (ii) a generative module that synthesizes this information to produce coherent and contextually accurate text (Lewis et al., 2020b). By combining these capabilities, RAG models are able to generate not only fluent and human-like text but also outputs that are grounded in up-to-date and factual data, significantly enhancing their reliability and applicability in real-world scenarios. We categorized existing RAG methods into the following two main classes according to whether training is needed for further discussion: training-free approaches and rraining-based approaches.
Training-free RAG. Training-free RAG methods address the challenges of frequent fine-tuning and updating model parameters, which require substantial computational resources and time (Lewis et al., 2020b). These approaches leverage retrieved knowledge directly at inference time by incorporating the retrieved text into the prompt, eliminating the need for additional training. As the performance of large language models (LL.Ms) is highly sensitive to input queries, many frreeng-free RAG methods refine prompts by integrating external knowledge (Jiang et al., 2023; Li al., 2023; Kim et al., 2023; Ram et al., 2023; Trivedi et al., 2023; Wang et al., 2023). For example, In- In-Context RALM (Ram et al., 2023) augments the generation process by prepending retrieved documents to the original prompt without altering L.M parameters. IRCoT (Trivedi et al., 2023) enhances reasoning by interleaving chain-of- thouhtt (CoT) generation and retrieval, ensuring access to more relevant information across iterative reasoning steps. SKR (Wang et al., 2023) enables flexible utilization of both internal and external knowledge by guiding LLMs to decide whether a question can be answered based on internal knowledge before invoking the retriever. Despite their efficiency, training-free RAG methods often face limitations in optimizing the retriever and generator for specific downstream tasks, leading to suboptimal utilization of retrieved knowledge. To address this, training-basee baaed RAG approaches fine-tune both components, enabling large language models effectively adapt and integrate external information.
Training-based RAG. This kind of methods aim to optimize both the retriever and generator to enhance their alignment and effectiveness. One typical success is DPR (Karpukhin et al., 2020), which employs two independent BERT (Devlin et al., 2019) encoders for queries and passages and trains them via contrastive learning. Ren et al. (2023) employs a two-stage approach, starting with the pretrain S-BERT (Reimers & Gurevych, 2019) as retrieval backbone, enhanced by an adaptive hybrid strategy to effectively gather relevant demonstration. Next, T5 model is used the generator, which is further fine-tuned align with the target labels and inputs. In contrast, RA-DIT (Lin et al., 2024) first fine-tuning LLMs to effectively use retrieved knowledge and then refining the retriever to align with the model's requirements. To address indiscriminate retrieval and the incorporation of irrelevant passages, Self-RAG (Asai et al., 2024) introduces special tokens to dynamically assess the necessity of retrieval and control its behavior. More recently, MemoRAG (Qian et al., 2024) incorporates a memory module that generates context-specific cues to link the knowledge base to precise information, improving retrieval accuracy and response quality.
Despite their advantages, RAG models rely heavily on the quality and relevance of the retrieved knowledge, as inaccuracies or irrelevant information can directly compromise the quality of the generated output. Furthermore, the dual-step process of retrieval and generation for each query introduces significant computational overhead, posing challenges for real-time and resource-constrained applications.
15

---

Test-Time Learning for Large Language Models
Table 6. Components of AdaptEval.
Category
Dataset
Sources
Geosignal
Geoscience knowledge base, etc. Agriculture data
Agriculture-AA
DomainBench
GenMedGPT-5k
ChatGPT-generated data
Wealth-alpaca Iora
FIQA data, etc.
Dolly-15k
Databricks data
InstructionBench
Alpaca gpt4
GPT-4 instruction data User instruction data
InstructionWild
GSM8K
OpenAI data
ReasoningBench
MetaMathQA
Advanced math datasets
Logiqa
Civil service logic tests
A.3. Test-Time Adaptation
Test-time adaptation (TTA) aims improve a model's performance on unseen test data, which may undergo distribution shifts, by learning directly from the test data during the testing phase. Based on their reliance backward propagation, categorize the related TTA works into the following two groups for discussion.
Backpropagation (BP)-Based TTA. A foundational approach in this category is Test-Time Training (TTT) proposed by (Sun et al., 2020). TTT involves training a source model using both supervised and self-supervised objectives during the training phase. At test time, the model is adapted using supervied objectives such as rotation prediction (Sun et al., 2020), contrastive learning (Liu et al., 2021; Bartler et al., 2022), or reconstruction learning (Gandelsman al., 2022). To address scenarios where modifying the training process or accessing source data not feasible, Fully TTA methods directly update pre- mre-trained models during testing. These methods rely on unsupervised learning objectives, with entropy minimization (Wang et al., 2021; Niu et al., 2023) emerging as one of the most widely used techniques due to its simplicity and effectiveness. Entropy minimization encourages the model to produce confident predictions by reducing uncertainty in its output distribution. This approach effectively aligns predictions to a single class. Beyond entropy minimization, other unsupervised objectives, such as prediction consistency maximization (Zhang et al., 2022a; Fleuret et al., 2021) and feature distribution alignment (Mirza et al., 2023), have also been explored, enhancing the model's 's ability to adapt to diverse test-time scenarios.
To further enhance the efficiency of backpropagation- based TTA, recent research efforts have focused on tow primary aspects: (1) Sample Efficiency: As not all test samples contribute equally to adaptation. Several recent works (Niu et al., 2022a; 2023; Shu et al., 2022; Lee et al., 2024) have introduced sample selection strategies to focus on reliable and non-redundant samples, reducing the noise in the gradient and the number of samples for TTA, therehy enhancing adaptation performance and efficiency. (2) Memory Efficiency: Addressing the memory-intensive nature of backpropagation, methods such as EcoTTA (Song et al., 2023) optimize parameter-effiiient components during adaptation, while MECTA (Hong et al., 2023) reduces batch size to lower memory consumption. Additionally, MECTA introduces a domain-aware batch normalization layer to stabilize model updates, even with smaller batch sizes. Similar the sample efficiency methods, we propose Sample-Efficient Learning Strategy that uses perplexity- weighting scheme to prioritize high-perplexity test samples for backpropagation, ensuring efficient utilization of computational resources.
The development of BP-free TTA has seen significant progress, with early research focusing primarily on the adjustment of batch normalization (BN) layer statisties using test data. These methods primarily involved recalculating the mean and variance of BN layers based on testing data (Nado et al., 2020; Schneider et al., 2020). However, such approaches were limited to adapting BN layers, restricting their applicability to architectures that heavily rely on BN. To overcome these limitations, more generalized methodologies have been proposed to enhance the flexibility and effectiveness of BP-free TTA. For instance, reconstruction-basee approaches focusing on input-level adaptation leverage advanced techniques like diffusion models to preprocess corrupted test images before prediction (Gao et al., 2023; Oh et al., 2025). Additionally, output-leve level adaptation methods have been developed, such as T3A (Iwasawa & Matsuo, 2021), which utilizes prototype-basee based classifier adjustment for adaptive predictions, and LAME (Boudiaf et al., 2022), which directly corrects the predicted logits. The recent advanced FOA (Niu et al., 2024) adapts models to unseen test samples without
Backpropagation-Free TTA.
16

---

Test- Time Learning for Large Language Models
Geogruphy
Logiqa
Agricultrre
MelaMath
ReasonIngRench
DomainBench
Medicine
GSMSK
Instruction Bench
Finance
InstructionWild
Dolly
AlpacaGpr4En
Figure Distributions AdaptEval.
backpropagation by learning a prompt through a derivative-free covariance matrix adaptation strategy and adjusting model activations to align with the source training domain. Looking forward, there great potential to extend our method to the realm of efficient BP-Free TTA, thereby further broadening the practical applicability our approach in diverse real-world scenarios.
B. vva Benchmark
To the best of our knowledge, no existing benchmark is specifically designed to evaluate the adaptability of Large Language Models (LLMs) across diverse data distributions. Since the diversity of tasks and domains inherently captures variations in data distributions, we address this gap by introducing a comprehensive benchmark, AdaptEval, which spans wide range of tasks and domains thoroughly assess the effectiveness of our proposed TLM. AdaptEval designed to capture two primary types of out-of-distribution (OOD) scenarios test time: vertical domain shift and distributional shift in non-specific domains, described in the previous section. To build diverse and challenging evaluation framework, collect high-quality datasets from HuggingFace, ensuring coverage across various data distributions. Specifically, AdaptEval consists of three categories of datasets: DomainBench, InstructionBench, and ReasoningBench. These categories are tailored to evaluate LLMs' adaptability to tasks requiring vertical knowledge, instruction-folloiing capabilities, and logical reasoning under distribution shifts. summary of the datasets included in AdaptEval is presented in Table 6, with further analysis provided below.
AdaptEval consists of the following three categories, shown in Figure 4.
DomainBench. This category includes four vertical domain knowledge datasets: Geography, Agriculture, Medicine, and Finance. It evaluates the adaptability of LLMs to specialized fields by assessing their ability to handle tasks requiring domain-spenn specific expertise, such as named entity recognition, judgment, and question answering. By incorporating domain-speciffe terminology and real-world complexities that may challenge model performance, DomainBench provides rigorous evaluation of models' proficiency in mastering and applying specialized knowledge.
InstructionBench. This category comprises three general- and InstructionWil. It evaluates the adaptability of LLms to instruction- based tasks by assessing their ability to comprehend, interpret, and execute diverse range of user instructions. The datasets cover various task types, such as question answering, text classification, and summarization, while introducing variations in user intent, phrasing, and
cnltrcction-olllwviii datasets: Alpaca-GPT4, Dolly,
17

---

Test-Time Learning for Large Language Models
800
Agriculturee
Agriculture
800
Medicine
700
Medicinee
Geography
Geography
Finance
Finance
600
Deasity
Density
500
400
400
500
200
200
100
0
50
100
150
200
0
100
200
300
400
Input Sequence Length
Output Sequence Length
Figure 5. Distribution Sequence Lengths for Samples DomainBench.
linguistic styles, providing thorough assessment of the model's capacity to process and respond effectively to diverse instructions in real-world scenarios.
ReasoningBench. This category contains three cessonninn-foussee datasets: GSM8k, MetaMath, and Logiqa, designed to evaluate the logical reasoning and math problem-solving abilities of LLMs. It evaluates the model's ability to handle intricate reasoning processes through diverse scenarios, including multi-step mathematical reasoning, complex math problem solving, and logical reading comprehension. ReasoningBench evaluates tasks that require precise and consistent reasoning, offering thorough test of model's ability to tackle complex problems and produce logical, accurate solutions.
B.1. DomainBench
DomainBench focuses on evaluating the model's adaptability and performance in four vertical domains: Geography, Agricultural, Medical, and Financial. To ensure the comprehensiveness and scientific rigor of the evaluation, Domain integrates four meticulously selected datasets: GeoSignal, Agriculture- GenMedGPT-5K, and Wealth-Alpaca_L. Each dataset is sourced from a broad range of specialized domains, enabling the measurement of large model performance on complex -emmai--ppecffic knowledge and task execution. The distribution of sequence lengths for the dataset samples and an example table of dataset entries are provided in Figure 5 and Table 7.
Geography: The GeoSignal' dataset a knowledge-intensinn instruction-tumie resource tailored for the Earth Sciences domain, aiming to improve model performance this field. It comprises approximately 39.7k 7k samples, created through a mix of human curation and semi- aumiaautomatic methods. The dataset is designed to align with user intent, featuring two sections: general section for human instruction alignment and professional section focused Earth Sciences expertise. It includes tasks such Named Entity Recognition (NER), relation inference, fact verification, and question answering, enriched with domain-specitn terminology like "volcanic neck" and "geomagnetic elements". Data sources span multi- Earth Sciences knowledge graph (GAKG), academic resources (DDE Scholar), and various databases and QA platforms. From this dataset, we randomly select 5k samples to form the Geography dataset, which evaluates the model's domain knowledge and task performance in Geography.
The Agriculture- QA² dataset focuses on agricultural QA, containing about 22.6k samples. It covers various aspects of agricultural production, such crop cultivation, livestock farming, soil management, and farming practices. These QA tasks challenge the model's ability to transfer knowledge and comprehend natural language, especially within the highly specialized agricultural field. We randomly select 5k samples to create the Agriculture dataset, aiming to evaluate the model's performance in handling agricultural QA tasks.
Agriculture:
hhtps://huggingfacecco http:/ggggn..geeeee/aaaaeednnann hhupss//hugginnffce.
http://whwggnnnnaaacccc///snssns wwwdatasets// www.aaawwwaaaaiiiiaaaaauanaaaaaa
18

---

Test-Time Learning for Large Language Models
Table 7. Examples of BemainRench data.
Input: wonder about the fossil of the Odonata. Can you help?
Geography
Output: Certainly, the Odonata has a fossil of Odonata- Wolfe et al. 2016.
Input: Why do cassava propagated through stem cuttings mature faster compared to seed propagation?
Agriculture
Output: This is because the stem cuttings already have head start in development and can quickly establish roots and start growing.
Instruction: If you are a doctor, please answer the medical questions based on the patient's description. Input: Doctor, I'm experiencing some strange symptoms. I've been acting hostile towards my family, and I'm having problems with my movement. Also, I've noticed that I've been slurring my words lately. Output: Based on what you' 've described, sounds like you may be experiencing delirium, which is state of mental confusion that can cause changes in behavior, movement problems, and difficulty speaking clearly. We'll 'Illl need to run some tests to determine the underlying cause, but it's important that we act quickly to manage your symptoms and prevent any complications.
Medicine
Input: Assemble an outfit for a Halloween party.
Finance
Output: For a Halloween party, assemble an outfit that will make you stand out! Start with a base layer of black trousers and a white t-shirt. Then pair it with an eye-catching statement piece, such as a bright orange blazer. For shoes, choose something with spooky vibe, like pair of lace-up boots with gothic buckles. Accessorize with deep red scarf and black fedora. Finish off the look with classic cane and golden pocket watch.
Medicine: GenMedGPT-5k3 with a total of 5.45k samples is medical dialogue dataset generated by ChatGPT, and is designed to emulate real-life conversations between patients and doctors. It provides rich, spenario-specific content that closely mirrors real medical scenarios. The dataset encompasses a wide range of medical queries from patients and responses or diagnoses from doctors. We randomly select 5k samples to create the Medicine dataset, aiming to assess the model's performance in medical dialogue, particularly its adaptability to medical contexts and the quality of its responses.
The Wealth-Alpaca_LLnn Lora* dataset is focused on the financial domain, consisting of 44.3k samples. It integrates general task data (Alpaca dataset), financial domain data (FiQA dataset), and custom task data generated using GPT-3.5. This dataset is extensively used for sentiment analysis, opinion mining, and QA tasks in financial texts, covering a variety of real-world applications in finance. We randomly select 5k samples to create the Finance dataset, designed to evaluate the model's specialized performance in financial QA tasks.
Finance:
B.2. InstructionBench
InstructionBench aims to assess the adaptability and performance of models across a diverse range of general instruction tasks, including, but not limited to, question answering (QA), text summarization, and classification. This benchmark integrates three carefully curated high-quality datasets: Alpaca-GPT4, Dolly, and Instruction encompassing a variety of instruction tasks generated through both human and model-driven approaches. The evaluation is designed to be both comprehensive and rigorous. The distribution of dataset samples and an example table of dataset entries are provided in Figure and Table
Dolly: The 15lly-15k dataset, created by Databricks, consists of 15k high-quality, human-generated prompt-response pairs. It is specifically designed for the instruction fine-tuning of large language models. Unlike datasets generated through model outputs or copy- sopy-pasting, Dolly-15k maintains authenticity and high quality by relying solely on human input. The dataset encompasses common instruction fine-tuning tasks, including QA, summarization, and classification. We follow the official guide from D and concatenate the sample fields into complete training samples. A subset of 5k samples is randomly selected to evaluate model performance.
Alpaca-GPT4: The Alpaca-GPT44 dataset comprises 52k fostowcii samples generated using GPT-4. The
httpss//wnggnggae....ooatsssssaa colaatssssspphhatt//aaalaaalaaee lora
"httss//huggingfaceecc
"hups://githuh.cm "htpp://gituh..om//aaaaa hwwpp//wwww.a.aaccoaaaa/o////aaa
hwwpgh/www...coteceoooaaaaaaanna gpt4 en
19

---

Test-Time Learning for Large Language Models
700
AlpacaCipetEn
AlpucaCipeIEn
Delly
Dolly
600
Instruction Wild
InstructionWild
500
800
Deasity
Density
400
600
300
400
200
200
100
0
20
40
60
80
100
120
140
0
100
200
300
400
Input Sequence Length
Output Sequence Length
Figure Distribution Sequence Lengths for Samples in InstructionBench.
Datascts
5000
Dalascts
GSMSK
GSMBK
Logiqu
Logiga
MetaMath
MetaMath
Deasity
250
Demity
3000
200
150
2000
100
1000
50
0-
0
12
Inpuli Sequenoe
Length
Output Sequence Length
Figure 7. Distribution of Sequence Lengths for Samples in ReasoningBench.
dataset is constructed by first manually creating comprehensive set of instructions across a wide range of tasks, followed by data generation and quality assurance using GPT-4. It includes diverse task types, such as various QA and summarization tasks. From this dataset, we randomly select 5k samples to test the model's generalization capabilities and performance on instruction tasks.
InstructionWike Wild: InstructionWild" is large dataset focused on real-world instructions, consisting of over 50k high- quality bilingual (Zh-En) instructions. The data is derived from real user-shared scenarios, offering diversity and broad applicability. Its format aligns with that of the Alpaca dataset, enabling seamless integration and usage. The dataset covers common instruction fine-tuning tasks such as QA and summarization. We randomly extract 5k samples to evaluate the model's ability to understand and execute instructions effectively.
B.3. ReasoningBench
ReasoningBench is designed to evaluate models' logical reasoning and problem-solving abilities through tasks such as mathematical problem solving, multi-step reasoning, and logical reading comprehension. This benchmark integrates three high-quality reasoning datasets, GSM8K, MetaMath, and LogiQA, to comprchensively assess models' reasoning performance across diverse dimensions and task types. The distribution of dataset samples and an example table of dataset entries are provided in Figure 7 and Table 9.
20

---

Test-Time Learning for Large Language Models
Table 8. Examples of InstructionRench data.
Instruction: Below is an instruction that describes a task. Write response that appropriately completes the request.
Dolly
Input: What athlete created the "beast quake' for the Seattle Seahawks?
Response:
Output: Marshan Lynch
Input: Imagine you are talking to a child, explain the concept of DNA.
Alpaca-GPT4
Output: Okay kiddo, so you know how we all have things that make us special and unique? Like how you have your mom's eyes and your dad's hair? Well, that's because of something called DNA. DNA is like the directions the recipe that our bodies follow to make who we are. It decides things like the color of our eyes, how tall we'll grow, and even how our personality is. S.Annnn Just like you follow instructions to build a Lego castle, the cells in our body follow the DNA's instructions to make us who we are. You might also hear DNA being called our "genetic code," because it's kind of like the secret language that our bodies use to build us. from our parents. So, we get half of our DNA from our mom and half from our dad. That's why we can have traits from both of them. But even though we share DNA with our family, while there is nobody else in the world with the exact same DNA
An n \n And the cool thing about DNA is that it's passed down
We are all unique and special in our own way!|m)), Isn't that cool? Do you have any more question about it?
Input: Create a list of five unique travel destinations that are not popular tourist spots. Provide a brief description for each destination, highlighting what sets it apart from other popular travel locations. Output: Meghalaya, India: Located in the northeast part of India, Meghalaya is known for its beautiful rivers, ancient caves, and rich cultural heritage. It' 9s one of the least explored regions of the country, making it an ideal destination for those looking for an escape from the crowds of popular tourist spots. |n Atacama Desert, Chile: The driest desert in the world, the Atacama Desert is other- landscape of stunning terrain and amazing geological formations. It) 0 19s home to some of the clearest skies on the planet and offers incredible stargazing opportunities, \n Antarctica: For the adventurous traveler, Antarctica is wild, unforgiving landscape like no other. It',22199 9s home to breathtaking glaciers, unspoiled coastlines, and an abundance of wildlife. With few visitors, ut)u2015 9s a paradise for those who want to explore the continent u2019s 9s remote beauty. An Georgia: Located between Europe and Asia, Georgia a little-known country full of culture, history, and stunning landscapes. From the city of Tbilisi to the mountains of Svaneti, there are endless opportunities for exploration and discovery. An Australian Outback: Few tourists venture to the remote stretches of the Australian Outback. but it' u2 9s well worth the effort. Here, you't (u2011100 01 9l1 find miles of unspoiled nature, rooted in Indigenous history and culture, and the chance to experience a whole different side of the continent.
Instruction Wild
GSMSK: GSM8K' is high-quality dataset of linguistically diverse elementary school math word problems, constructed by OpenAI. It contains a total of 8.5k samples. The dataset's 's unique feature lies its human-craftee problems, avoiding templated language, and providing natural language solutions instead of purely mathematical expressions. This design ensures closer alignment with real-world problem-solving scenarios. The primary task type is multi-step mathematical reasoning, which effectively diagnoses deficiencies in a model's reasoning capabilities. Consistent with common pratice, we apply a zero-shot bhain-of-thougee (CoT) prompt to each sample, guiding the model to think step by step. For evaluation, we combine the training and test sets and randomly select 5k samples.
MetaMath: MetaMath 10 is large-scale dataset comprising approximately 395k samples, designed to enhance mathematical reasoning through grestion-guidee approach. It diversifies mathematical problems by rephrasing and restructuring them from multiple perspectives, offering a robust and challenging benchmark for evaluating mathematical reasoning abilities. The dataset focuses on QA tasks and spans broad spectrum of mathematical problem complexities. Similar to GSM8K, we apply a zero-shot CoT prompt to each sample to guide the model in logical reasoning. For evaluation purposes, we randomly select 5k samples from the training set.
Phops://huggingfacecoom http://www........a.ccco///nnnnn http://ggggnna.ccoooooaaaeeeenna
21

---

Test-Time Learning for Large Language Models
Table Examples of ReasoningRench data.
Input: Below is an instruction that describes a task. Write a response that appropriately completes the request.
GSM8K
Instruction: (nnie food companies sponsored a local food bank. Foster Farms donated 45 dressed chickens; American Summits donated twice the number of bottled water than the number of dressed chicken donated by Foster Farms; Hormel donated three times the number of dressed chickens that Foster Farms donated; Boudin Butchers donated one-third of the number dressed chickens that Hormel donated; Del Monte Foods donated 30 fewer bottles of water than American Summits. How many food items did the companies donate in total?
Response: Let's think step by step.
Output: 375
Input: Below is an instruction that describes a task. Write response that appropriately completes the
MetaMath
request. Instruction:) What is the value of the ceiling function applied to 49 \n Response: Let's think
step by step.
Output: -3
Input: Write multi-choice question for the following article:) n Article: Researchers believe that if mothers are exposed more pesticides in the first few months of pregnancy, the babies born may be less intelligent. They believe that the embryonic brain begins develop shortly after pregnancy, so the pre-pregnancy is the baby's s brain In the critical period of development, exposure to more pesticides may change the environment around the developing embryo's brain in pregnant women. \n Question: Which of the following, if true, would best support a researcher's point of view?' Options: A. Many babies are born early due to their mothers' exposure to pesticides. \n B. Insecticides are a potential threat to people's health, and it can also cause many diseases such as Parkinson's 's disease, cancer and mental illness. \n Previous research has found that increased exposure to pesticides can cause thyroid problems in pregnant women, and the thyroid status of pregnant women can affect the intellectual development of the fetus. \n D. Researchers conducted follow-up survey of 1,500 pregnant women and found that children born to pregnant women who were more exposed to pesticides performed significantly worse in mathematics and language. \n'in Answer:
LogiQA
Output: C
LogiQA: LogiQA" is a high-quality, comprehensive dataset focused on logical reasoning, derived from logical reasoning questions used in the Chinese National Civil Service Examination. It consists of 8k QA samples, covering a variety of deductive reasoning tasks designed to test a model's adaptability to logical reasoning and problem-solving. During dataset construction, strict filtering was applied to exclude samples with inappropriate formats or those involving charts and mathematical calculations, ensuring the dataset's purity and quality. Following the official dataset guidelines!? we create multiple-chicee prompts and randomly select 5k samples for evaluation.
C. More Details for Experiment Settings
C.1. More Metrics
To provide a more comprehensive evaluation of the proposed method, we employ additional evaluation metrics, including BERTScore (BS FI) (Zhang et al., 2019), BLEU (Papineni et al., 2002), Rouge-1 (R-1), Rouge-2 (R-2), and Rouge-L (R-L). These metrics help assess various aspects of model performance, such as the quality of generated text, its similarity to reference outputs, and the model's ability to capture key information at different levels of granularity.
C.2. Implementation Details
The training and evaluation are conducted on the 80G memory-sized NVIDIA A800 GPUs with CUDA version 12.1. Our method is implemented using the PyTorch framework with Pytorch, version 2.5.1. The training framework used is LLaMA-Factory For the LoRA setup, we use random Gaussian initialization for matrix A and set matrix zero, with
22

---

Test-Time Learning for Large Language Models
Table I0. Comparison of experimental results on Geography dataset DomainBench.
Method
BERTScore
BLEURT
BLEU
Rouge-1
Rouge-2 t
Rouge-L
Balma3.2-3B-Instteee
0.6905
-0.6794
0.0638
0.2661
0.1020
0.1917
Tent
0,6661
-1.1320
0.0444
0.2149
0.1149
0.1780
EATA
0.5274
-1.4276
0.0032
0.0065
0.0007
0.0064
TLM (Ours)
0.7160
-0.7278
0.0952
0.3203
0.1526
0.2534
Llama3--B-IIsstme
0.6953
-0.6161
0.0709
0,2711
0.1040
0.1977
Tent
0.6007
-1.5575
0.0087
0.0932
0.0100
0.0756
EATA
0.6751
-1.0290
0.0575
0,2432
0.1290
0.1949
TLM (Ours)
0.7284
-0.5860
0.1064
0,3540
0.1650
0.2835
[Jama2-13B-ol
0.6707
-0.7337
0.0495
0.2430
0.0975
0.1706
Tent
0,4541
-1.2410
0.0011
0,0321
0.0000
0.0320
EATA
0.6999
-0.7661
0.0703
0.3132
0.1441
0,2421
TLM (Ours)
0.6902
-0.6738
0.0722
0,2996
0.1319
0.2225
Qwen2.5-7B5 -7t-ttsrctt
0.7003
-0,5802
0.0823
0.2911
0.1125
0,2128
Tent
0.6925
-0.9422
0.0810
0,2703
0.1293
0.2242
EATA
0.6856
-0.9806
0,0723
0,2444
0.1182
0,2063
TLM (Ours)
0.7214
-0.5093
0.1039
0.3412
0.1481
0.2632
a rank of = 8. The LoRA is applied only to the Wg and W..
Tent (Wang et al., 2021) is a Test-Time Adaptation (TTA) method originally designed for image classification tasks, which adapts model's parameters during inference based on the entropy of predictions. In the context of dynamic parameter updates for Large Language Models (LLMs), we adapt the Tent method to work with LLMs by leveraging the prediction entropy of the 80 tokens generated by the model. Specifically, we calculate the entropy of the model's predictions for these 80 tokens during each test-time update and use this information to adjust the LLM's parameters.
EATA (Niu et al., 2022a) is state-of-the-ee TTA method for image classification models, which adjusts model parameters based on low-entropy samples during inference. In the context of dynamic parameter updates for LLMs, we adapt the EATA method by leveraging the prediction entropy of 80 tokens generated by the model to select samples. Specifically, during each test-time update, we compute the entropy the model's predictions for these 80 tokens and, following the setup of EATA, adjust the parameters of the LLM (with the hyperparameter Es of EATA set 0.4). However, performing EATA updates on Jlama2-13B-chi may result in out-of-memory errors. To address this issue, reduce the number of tokens from 80 to 30 when applying EATA updates Llama2- 1anta2--1BBchh 13B-chat.
Offline and Online Settings. In our experiments, we consider two distinct settings: Offline and Online. In the Offline setting, all test data is processed at once, and the model's parameters are updated using all available test samples before any testing is performed. In the Online setting, test data sequentially, where the model is updated after each individual test sample or batch. This setting better reflects real-world scenarios where data arrives in continuous stream, requiring real-time updates to the model.
D. More Results of Experiment
To comprehensively evaluate the effectiveness of the proposed TLM, we report additional evaluation metrics and results. As shown in Table 10, the proposed method outperforms both the cannooy-mmiiiizttinn-baaaee method (Tent) and the original LLMs on the Geography dataset. Specifically, the proposed method achieves a 4.76% improvement in BERTScore compared to Llama3-8B-Instrus As shown Table 11, our proposed TLM outperforms Tent on the Agriculture dataset. For instance, our proposed TLM achieves a 146.61% improvement the BLEU metric compared to Qween2.--BB-nsttuus From Table 12, the proposed method outperforms the original LLM the Medicine dataset. Specifically, compared to Maana--8B-Instruee the proposed method achieves relative improvement of 7.05% in BERTScoree. As shown in Table 13, our proposed TLM outperforms Tent on the Finance dataset. For instance, our proposed TLM achieves a relative improvement 122.50% improvement the BLEU metric compared to -llama..2-3BIIssmmte Instrruct From Table 14, the proposed method outperforms the original LLM on the Alpaca-GPTI dataset. Specifically, compared Llama3-8B. -----nnItuctt the proposed method achieves relative improvement of 4.30% BeRTScore. As shown in Table 15, the proposed method
23

---

Test-Time Learning for Large Language Models
Table 11. Comparison experimental results on the Agriculture dataset DomainBench.
Method
BERTScore
BLEURT
BLEU
Rouge-1 1
Rouge-21
Rouge-L
Llama3.2-3B 3-B-I.--ttt
0.6672
-0.7595
0,0104
0.0927
0.0316
0.0684
Tent
0,6350
-0.9975
0.0015
0.0159
0.0034
0.0148
EATA
0.6659
-0.8355
0.0022
0.0263
0.0046
0.0247
TLM (Ours)
0.6787
-0.6668
0.0288
0.1935
0.0655
0.1518
Llama3--B-Innsime
0,6666
-0.7288
0.0106
0,0903
0.0328
0.0665
Tent
0.5746
-1.2577
0.0008
0.0075
0.0011
0.0066
EATA
0,5870
-1.3074
0.0003
0,0019
0.0000
0.0016
TLM (Ours)
0.6526
-0.6703
0.0189
0.1477
0.0496
0.1110
JJama2-13B-cl
0.6235
-0.6134
0.0116
0.0910
0.0319
0.0668
Tent
0,5901
-0.9157
0.0015
0,0200
0.0019
0.0193
EATA
0.6065
-0.6084
0.0101
0.0843
0.0257
0,0675
TLM (Ours)
0.6354
-0,5929
0.0144
0.1108
0.0379
0.0826
Qwen2.5-7B --n--stttt
0.6562
-0.7226
0.0118
0.1068
0.0377
0.0768
Tent
0.6667
-1.1641
0.0118
0.1399
0.0346
0.1145
EATA
0.6562
-1.1517
0,0113
0.1425
0.0357
0.1166
TLM (Ours)
0.6755
-0.6219
0.0291
0.1841
0.0663
0.1394
outperforms both Tent and the original L.MSs across all metrics on the Dolly dataset. Specifically, the proposed method achieves relative improvement of 1.1 1.11% in BERTScore compared to Llama3-8B-Instruet. As shown in Table 16, our proposed TLM outperforms Tent the Instruction Wild dataset. For instance, proposed TLM achieves relative improvement 15% improvement in the BLEU metric compared to Llama3.2- Bettn...---BImaee I-sttruttt...
E. Discussions and Future Works
To the best of our knowledge, addressing the challenges faced by LLMs real-world deployments, such as distributional shifts in test data, has not been thoroughly explored. this work, we introduce the Test-Time Learning (TTL) task, aiming dynamically adapt LLMs using only unlabeled test data during testing. Additionally, propose the AdaptEval benchmark, designed evaluate the effectiveness of TTL in enhancing model performance across variety of domains. Experimental results demonstrate that the proposed Test-Time Learning approach effectively improves LLM performance on target domains. However, we believe there are several potential research directions worth exploring in the future:
Cross-Domait Continuous Adaptation: When deploying LLMs across multiple domains, it is essential to achieve continuous adaptation without overfitting to specific domain. This requires balancing the transfer of knowledge across domains while mitigating catastrophic forgetting. Future work could explore methods for seamless domain adaptation that improves model generalization across dynamic and diverse tasks.
LLMs have large parameter sizes, and deployed models in practical settings may not support backpropagation due to memory or computational limitations. This restricts the ability to perform Test- Time Learning during inference, highlighting the need for more efficient methods that enable model adaptation without requiring backpropagation.
Only Forward Passes:
24

---

Test-Time Learning for Large Language Models
Table 12. Comparison of experimental results on the Medicine dataset of DomainBench.
Method
BERTScore
BLEURT
BLEU
Rouge-1 f
Rouge-2 f
Rouge-L
Llama3.2-33 1anaa3.2-BBBha 2-suutnsttt
0.6677
-0.7668
0.0165
0,1588
0.0269
0.1037
Tent
0.6322
-1.2819
0.0191
0.1897
0.0276
0.1204
EATA
0,6554
-0.9669
0.0018
0,0312
0.0017
0,0178
TLM (Ours)
0.7005
-0,5603
0.0463
0.2559
0.0728
0.1844
Llama3 8B 3B-Instruct
0.6628
-0.7486
0.0136
0.1398
0.0343
0.0911
Tent
0,5525
-1.3487
0.0014
0.0113
0.0006
0.0089
EATA
0.6072
-1.1079
0.0011
0.0139
0.0011
0,0100
TLM (Ours)
0.7095
-0.4781
0.0486
0.2646
0.0836
0.1889
Llama2-133 1-1hhah-
0,6559
-0.5971
0,0158
0.1439
0.0397
0.0956
Tent
0,6235
-0,5867
0.0100
0.1250
0.0261
0,0874
EATA
0.6543
-0.4349
0.0145
0.1465
0.0410
0.1007
TLM (Ours)
0.6988
-0.4615
0.0512
0,2370
0.0890
0.1760
Swen2.5--B-Innstee Instrructtt
0.6561
-0.6598
0.0136
0,1441
0.0369
0.0909
Tent
0.5884
-1.1911
0.0097
0.0548
0.0214
0.0454
EATA
0.6408
-0.7171
0.0163
0.1672
0.0304
0.1202
TLM (Ours)
0.7082
-0.5964
0.0623
0.2623
0.1003
0.1967
Table 13. Comparison of experimental results on the Finance dataset of DomainBench.
Method
BERTScore
BLEURT t
BLEU
Rouge-1 f
Rouge-21
Rouge-L
Llama3.2 1aamaa..233B--a Inssstrruttt
0.6809
-0.6532
0.0360
0,2425
0.0791
0.1602
Tent
0.6322
-1.2819
0.0191
0.1149
0.0549
0.1084
EATA
0.5028
-1.2700
0.0010
0.0150
0.0001
0.0149
TLM (Ours)
0.7060
-0.5113
0.0801
0.3206
0.1255
0.2367
JJama33-8-Innttem
0.6862
-0.6230
0.0407
0.2530
0.0873
0.1682
Tent
0,5919
-0.9704
0.0032
0.0390
0.0072
0.0351
EATA
0.6431
-1.2160
0.0149
0.1398
0.0567
0.1183
TLM (Ours)
0.7153
-0.4411
0.0952
0,3514
0.1448
0.2576
Llama2- 13B-chat
0.6814
-0.6217
0.0427
0.2591
0.0892
0.1722
Tent
0,4878
-1.1973
0.0006
0,0049
0.0000
0.0049
EATA
0.6203
-1.0560
0.0250
0.1241
0.0574
0.1113
TLM (Ours)
0.6979
-0.5110
0.0628
0.3009
0.1104
0.2064
Qwwn2.5-78-Innee
0.6978
-0.5370
0.0639
0.2969
0.1124
0.2025
Tent
0,6623
-1.1316
0.0313
0.1835
0,0845
0.1533
EATA
0.7103
-0.5746
0.0810
0,3127
0.1358
0.2409
TLM (Ours)
0.7220
-0.3890
0.0968
0.3607
0.1487
0.2616
25

---

Test-Time Learning for Large Language Models
Table 14. Comparison of experimental results on the Alpaca-GPT4 dataset of InstructionBench.
Method
BERTScore t
BLEURT
BLEU
Rouge-1 f
Rouge-2 t
Rouge-L
Llama3.2-33 233--ssruutt
0.7260
-0.5164
0.0953
0,3885
0.1712
0.2619
Tent
0.5656
-1.5454
0.0054
0.0351
0.0032
0.0305
EATA
0,6553
-1.2411
0.0220
0.1513
0.0753
0,1359
TLM (Ours)
0.7523
-0.4148
0.1239
0.4170
0.2062
0.3184
Llama3 8B 3B-Instruct
0.7353
-0,4655
0.1082
0,4076
0.1863
0,2796
Tent
0,6712
-1.0652
0.0421
0.2193
0.1193
0.1881
EATA
0.6398
-0.8220
0.0271
0.1499
0,0725
0.1332
TLM (Ours)
0.7669
-0,3090
0.1479
0,4582
0.2349
0.3528
Llama2- -3B-hatt
0.7326
-0,5290
0.1091
0.4068
0.1830
0,2777
Tent
0,6249
-1.9933
0.0108
0,0998
0.0444
0,0937
EATA
0,5073
-0.7398
0.0094
0.0813
0.0191
0.0799
TLM (Ours)
0.7386
-0.5150
0.1229
0,4311
0.1987
0.3040
Qwen2.5-7B 1recn..5-BBInsttruee 1--Instruct
0,7624
-0,2949
0.1556
0,4803
0,2353
0,3406
Tent
0.6787
-0.9809
0.0501
0.2323
0.1315
0,2031
EATA
0.0000
-1.9681
0.0000
0,0000
0.0000
0.0000
TLM (Ours)
0.7829
-0.2137
0.1749
0.4919
0.2648
0.3819
Table 15. Comparison of experimental results on the Dolly dataset of InstructionBench.
Method
BERTScore
BLEURT 1
BLEU
Rouge-1 f
Rouge-2
Rouge-L. L
Llama3.2 2--maB3.2-3B--ah Inss-trruttt
0.7289
-0.4824
0.0946
0.3778
0.1903
0.2935
Tent
0.6866
-0.5922
0.0797
0.2374
0.1398
0.2039
EATA
0,5767
-1.5690
0.0027
0,0092
0.0002
0.0086
TLM (Ours)
0.7334
-0.4783
0.1030
0.3878
0.1997
0.3048
Jlama33-B-Instten
0,7415
-0.4088
0.1114
0,4126
0.2137
0.3222
Tent
0,5905
-1.4648
0.0000
0.0036
0.0000
0.0036
EATA
0.6559
-1.2475
0.0275
0.1991
0,0868
0.1661
TLM (Ours)
0.7497
-0,3972
0.1194
0,4239
0.2226
0,3388
Llama2- 13B-chat
0.7074
-0.8015
0.0743
0.3274
0.1550
0.2407
Tent
0,4111
-0.7963
0.0008
0,0076
0.0000
0.0076
EATA
0.5355
-0.9809
0.0076
0.0632
0.0175
0.0577
TLM (Ours)
0.7108
-0,8169
0.0789
0.3411
0.1625
0,2516
Qwen2.5-7B InI-ttstt
0.7212
-0.4784
0.0911
0.3501
0.1790
0.2634
Tent
0,6674
-1.1697
0.0420
0.2373
0.1036
0.1051
EATA
0.6737
-1.1244
0.0491
0,2482
0.1132
0.1963
TLM (Ours)
0.7216
-0.5209
0.0931
0.3567
0.1832
0.2685
26

---

Test-Time Learning for Large Language Models
Table 16. Comparison of experimental results on the Instruction Wild dataset of InstructionBench.
Method
BERTScore
BLEURT f
BLEU
Rouge-1
Rouge-2
Rouge-L
Llama3.2-3B-IInttee
0,7019
-0.4837
0.0337
0,2796
0.0942
0.1699
Tent
0.5409
-1.2307
0.0018
0.0341
0.0008
0.0341
EATA
0.4537
-1,3202
0.0011
0.0123
0.0006
0.0122
TLM (Ours)
0.7044
-0.4503
0.0506
0.3104
0.1136
0.2004
0.7071
-0.4469
0.0363
0.2846
0.0976
0.1735
Tent
0,6092
-0,5488
0.0148
0,0933
0.0317
0.0910
EATA
0.6455
-0.5337
0.0133
0.0956
0,0452
0.1125
TLM (Ours)
0.7133
-0,3669
0.0554
0,3206
0.1207
0.2050
Llama2-13B-ca
0.7025
-0.4040
0.0464
0.3032
0.1062
0.1860
Tent
0,5390
-0.6001
0.0253
0.1294
0.0464
0,1260
EATA
0.5869
-0.7337
0.0159
0.1129
0.0295
0.0947
TLM (Ours)
0.7056
-0,3792
0.0499
0,3122
0,1119
0.1926
Qwen2.5-78 Instruct
0.7071
-0.4295
0.0436
0,3116
0,1070
0.1875
Tent
0.6865
-1.0060
0.0153
0.1829
0.1232
0.1615
EATA
0.6911
-0.9889
0.0157
0,1830
0.1233
0.1615
TLM (Ours)
0.7261
-0.2746
0.0813
0.3806
0.1498
0.2472
27


Reviews:
[{"review_id": "Gb4rYygLm5", "reviewer": "Reviewer_BSk7", "summary": "This paper proposes a test-time learning scheme for LLMs that minimizes input perplexity (instead of entropy in Test-Time Adaptation) for unlabeled data at test time for better domain adaptation. In addition, the authors draw an insight showing that high-perplexity inputs are more informative for optimization and thus propose to select those for test-time updates. Extensive empirical experiments suggest that the proposed method is generally effective.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">Q1. The claim \"This improved representation ..., thereby reducing conditional output PPL\" is not properly justified either theoretically or empirically.\n\n**A1.** We thank the reviewer for this important observation. We agree that the original statement regarding conditional output PPL reduction requires more precise formulation and supporting evidence. Please allow us to clarify:\n* **Causal Relationship Clarification.** We have revised the phrase \"thereby reducing conditional output perplexity\" to **\"This improved representation facilitates more accurate and confident next-token predictions, which is expected to reduce output perplexity\".** \n* **Theoretical Motivation Clarification.** In our study, we observe a strong positive correlation between input and output PPL (see Fig.1b). Relying on this, we seek to reduce the output PPL by input PPL minimization. \n\n* **Empirical Validation.** We compare changes in both input and output PPL between our method and the original LLM in Tab. A. **Our method reduces input PPL and consistently decreases output PPL.**\n\nTab. A: PPL results on DomainBench.\n|| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|-|\n|Input PPL|Llama3.1-8B|187.3|933.2|43.6|323.0|\n||Ours|6.6|3.9|2.0|5.5|\n|Output PPL|Llama3.1-8B|3242.9|205247.1|7.7|1208976.9|\n||Ours| 2483.9|156232.3|6.6|242617.0|\n\n>Q2. Add PPL as metric.\n\n**A2.** As suggested, we include PPL as a metric in Tab. B. **Our method outperforms existing methods in terms of PPL**.\n\nTab. B: PPL results on DomainBench. \n| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|\n|Llama3.1-8B|3242.9|205247.1|7.7|1208976.9|\n|EATA|4692.1|628196.9|2937.1|11444422.0|\n|Ours|**2483.9**|**156232.3**|**6.6**|**242617.0**|\n\n>Q3. The training/restoration setting is not clear. \n\n**A3.** In our method, the model weights are not restored to their original state after processing each test sample. Specifically,\n* **Offline Setting.** All test data are processed at once, and the model is updated using all available test samples before any testing.\n* **Online Setting.** Test samples are processed sequentially, with the model updated after each test sample or mini-batch, and the model parameters are never reset.\n\n>Q4. Differences of the proposed method from [R Sennrich]. \n\n**A4.** Both works use PPL minimization, and our method differs from [R Sennrich] as:\n* **Problem Setting.** We focus on **adapting LLMs at test time using unlabeled test data** to handle distribution shifts. In contrast, [R Sennrich] assumes access to **labeled source-domain training data** and optimizes translation models from a static source corpus. \n* **Optimization Objectives.** We adopt **input PPL minimization** as the optimization objective, enabling efficient self-adaptation of LLMs to target domains during testing. Instead, [R Sennrich] optimizes **output PPL** using labeled data.\n\n>Q5. Justification of batch setting needed.\n\n**A5.** We evaluate batch size in the **online setting** (bs=1 vs 100) under streaming-like scenarios. **With bs=1, the model updates after each test sample. Results demonstrate increased update steps（1514→2541), while our TTL remains effective (0.2040→0.1917).**\n\n>Q6&Q7. Differences between TTA and TTL? Also, PPL and entropy?\n\n**A6&A7.** Both TTA and TTL adapt models without labeled data, but they differ in key aspects:\n* **Objective Function.** Most TTA methods minimize the output entropy $H(P(y|x))$. TTL minimize the PPL of the input sequence $\\mathcal{P}(x)=e^{1/T\\sum_{t=1}^TH_{CE}^{(t)}}$, where $H_{CE}^{(t)}$ is the token-level cross-entropy.\n* **Task-Specific Adaptation.** TTA seeks to improve output confidence but cannot optimize the input representation quality that is essential for LLMs. In contrast, TTL minimizes input PPL to refine the internal representation of the input sequence to encourage more accurate predictions.\n\n>Q8. Can you show a comparison of Fig.1b, which presents an updated input and output PPL after input PPL minimization? \n\n**A8.**  Fig. 1(b) shows **a strong positive correlation between input and output PPL** on DomainBench using Llama3.1-8B with varying degrees of training. We compare the changes in both input and output PPL of our method with the original LLM in Tab. A, **our method reduces input PPL and consistently decreases output PPL**.\n\n>Q9. Fig.1c meanings and Training Protocol. \n\n**A9.** Clarifications are below:\n* Fig. 1c shows how the selection of different proportions of test samples (X-axis: top/bottom p% by PPL) would impact the final performance  for TTL  (Y-axis: evaluated on the full test set).\n* **Training Protocol.** We pre-sort test samples by PPL, select a given proportion, **randomly shuffle the subset**, and perform TTL. The adapted model is then evaluated on all test data.\n****\nWe sincerely hope our clarifications above have addressed your concerns. We would be grateful if you could kindly reconsider the evaluation of our paper.", "claims_and_evidence": "In Sec. 4.1, the authors claimed \"This improved representation facilitates more accurate and confident next-token predictions, thereby reducing conditional output perplexity\", which is not formally derived or supported. After the minimization of input perplexity, the model parameter $\\Theta$ has changed, thus it is different from the real conditional output perplexity $\\mathcal{P}(y | x; \\Theta)$. So, the claim is not properly justified either theoretically or empirically.", "methods_and_evaluation_criteria": "Rouge scores and accuracy using extract match make sense. But, since the paper uses perplexity minimization for optimization, the perplexity itself can be one meaningful metric which is not presented in the paper.", "theoretical_claims": "There are no proofs or formal theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes, I have checked Section 5 Experiments. The setting of model training or restoration is not specified well. For example, in a typical setting of Test-Time Training, the model will be reverted to its original weight after training of each test sample. However, this setting of model optimization is not described in the paper.", "relation_to_broader_scientific_literature": "The finding that minimizing perplexity is performing better than minimizing cross-entropy is relevant to the test-time adaptation literature and test-time training literature.", "essential_references_not_discussed": "Yes. The key contribution is a method that minimizes input perplexity for test time training/adaptation, but there was prior work [R Sennrich'EACL 2012] that considered perplexity minimization for domain adaptation. This could harm the novelty of the paper.\n\n[R Sennrich'EACL 2012] \"Perplexity Minimization for Translation Model Domain Adaptation in Statistical Machine Translation\"", "other_strengths_and_weaknesses": "## Other Weaknesses\n\n- The justification of batch setting is not discussed, as in real-world scenarios testing data often comes in a streaming fashion.\n- Although the key claim of this paper is to minimize perplexity for test input instead of minimizing entropy as prior works, the perplexity is just a monotonic transformation of entropy itself, i.e., $Perplexity(P) = e^{H(P)}$. Thus, minimizing cross entropy directly minimizes perplexity. It is not well-understood what the real difference is and how perplexity benefits the test-time adaptation.", "other_comments_or_suggestions": "- Line 161: the model “is” optimized to .. \"is\" should be added.", "questions_for_authors": "- What is the essential difference between TTA and TTL? Minimizing perplexity or entropy is almost the same.\n- Can you show a comparison of Figure 1b, which presents an updated input and output perplexity after input perplexity minimization? This is helpful for understanding the effectiveness of the correlation after TTL.\n- What does Figure 1c mean? Are low perplexity samples and high perplexity samples mixed together in the training or separate?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ypm3K0oTGI", "reviewer": "Reviewer_CEYm", "summary": "In this paper, the authors propose a new Test-Time Learning (TTL) approach called Test-Time Learning for LLMs (TLM) that uses unlabeled test data to address distribution shifts arising from specialized domains. They highlight three main contributions:\n\n1. Self-Supervised Perplexity Minimization: The authors set a perplexity-minimization objective to adapt LLMs at test time through a self-supervised approach.\n2. Sample-Efficient Learning Strategy: To achieve more efficient and effective parameter updates, they selectively utilize high-perplexity samples for training, thereby focusing on instances that provide the most useful information.\n3. LoRA for Stable Adaptation: They integrate LoRA to mitigate catastrophic forgetting and ensure stable model adaptation during test-time learning.\n\nFurthermore, the authors introduce a new benchmark called AdaptEval to comprehensively evaluate TLM. AdaptEval covers vertical domain knowledge, instruction-following tasks, and reasoning datasets, thereby assessing the utility of TLM under various real-world distribution shifts. Experimental results demonstrate that TLM substantially improves performance on data that are highly specialized or deviate from the training distribution compared to existing methods, making it a promising solution in real-world deployment scenarios where labeled data are scarce.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely appreciate your high level of encouragement for our work. Your recognition of \"provides **a solid rationale for the necessity of TLM** as a solution\", \"the paper **strongly justifies the efficacy of perplexity-based optimization**\", and \"AdaptEval appears likely to **serve as a unified standard for subsequent research**\" is deeply appreciated. Our detailed responses are below:\n\n>Q1. Although the proposed method achieves generally positive experimental results, there are no concrete examples of actual outputs included (only input/output examples of the datasets are mentioned), making it less clear how the model performs in real usage scenarios.\n\n**A1.** We sincerely thank you for highlighting the need for concrete examples to demonstrate practical efficacy. As shown in https://anonymous.4open.science/r/ICML1861, our method consistently outperforms SOTA methods across diverse scenarios. Our analysis is detailed below:\n* **More accurate reasoning steps.** In reasoning tasks, our method produces logically sound and well-structured intermediate steps, enabling the model to reach correct conclusions more reliably. \n* **Better coherence and fluency.** Compared to TTA methods, our method maintains output fluency and avoids the repetition issues commonly observed in models like Tent or EATA.\n\n>Q2. Recent TTA methods not discussed. Moreover, a direct comparison to fine-tuning and RAG approaches is missing, which is regrettable.\n\n**A2.** According to your suggestions, we further compare with a SOTA TTA method, COME (ICLR'25,[r1]). From the results in Table A, **our method consistently and significantly outperforms COME across all domains**. This further demonstrate the superior performance of our method in addressing LLMs adaptation to new domains using only test data.\n\n**Experimental Design Rationale with RAG and fine-tuning approaches.** Our experiments focus on TTL, which uses only unlabeled test data without relying on additional labels or retrieval resources. Fine-tuning typically depends on task-specific labeled data, while RAG assumes a well-maintained knowledge base that may not generalize across domains. In contrast, our method updates solely on unlabeled test data via perplexity minimization. Therefore, we compare with TTA methods that share similarly label-free, resource-free settings.\n\nTable A: Comparison between COME and our method on *DomainBench*.\n| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|\n|COME [r1]|0.0048|0.0039|0.0301|0.0328|\n| **TLM(Ours)** | **0.3212**|**0.1319**| **0.2372** |**0.3242**  |\n\n[r1] COME: Test-time Adaption by Conservatively Minimizing Entropy, ICLR 2025.\n\n>Q3&Q4. More detailed comparisons of time complexity and resource requirements would be welcome when handling different types of domains (e.g., conversational tasks, online streaming data). \n\n**A3&A4.** Thank you for the valuable comments. According to your suggestions, we add experiments under the **online streaming setting (batch size = 1)**, which more accurately reflect real-world inference conditions, such as conversational or sequential inputs. Detailed analyses are as follows:\n*  **Time Complexity.** In the TTL setting, total computation time $T_{\\text{total}}$ consists of forward passes plus backward updates, i.e., $T_{\\text{total}} = T_{\\text{forward}} + T_{\\text{backward}} \\cdot N_{\\text{selected}}$, where $N_{\\text{selected}}$ denotes the number of samples used for backpropagation. Thanks to our Sample Efficient Learning Strategy, our method significantly reduces $N_{\\text{selected}}$ compared to baselines. For instance, as shown in Table B, our method requires up to 45% fewer backpropagation than EATA, resulting in lower overall computation time.\n*  **Resource Requirements.** As shown in Table B, we adopt LoRA for parameter updates, thereby limiting the trainable parameters to a small fraction of the full model. This design keeps memory usage and computational overhead low, especially when compared to full-parameter tuning. Consequently, our method enables practical adaptation in resource-constrained environments.\n\nTable B. Experimental results of our proposed method in the **Online setting (batch size=1)**.\n| Methods|Average R-Lsum|Trainable parameters |#Backwards|\n|-|-|-|-|\n|Llama3.1-8B|0.1720|-|-|\n|Tent|0.1834|3.41M|5000|\n|EATA|0.1809|3.41M|4634|\n|Ours| **0.1917**|3.41M|**2541**|\n\n****\nWe sincerely hope our clarifications above have addressed your concerns. We would be grateful if you could kindly reconsider the evaluation of our paper.", "claims_and_evidence": "Strengths:\n  - The paper convincingly explains why large-scale foundation models struggle to adapt to specialized domains or changing distributions, and provides a solid rationale for the necessity of TLM as a solution.\n  - Through both theoretical and empirical evidence on the relationship between perplexity and autoregressive models, the paper strongly justifies the efficacy of perplexity-based optimization. In addition, it clearly shows how LoRA helps mitigate catastrophic forgetting.\n  - By utilizing the variety of datasets in the AdaptEval benchmark—including domain knowledge, instruction-following tasks, and reasoning tasks—the paper demonstrates performance improvements of the proposed method in multiple contexts.\n\nWeaknesses:\n  - Although the proposed method achieves generally positive experimental results, there are no concrete examples of actual outputs included (only input/output examples of the datasets are mentioned), making it less clear how the model performs in real usage scenarios.\n  - The paper compares its method mainly with Tent and EATA, yet there are many other recent TTA methods not discussed. Moreover, while earlier in the paper it references fine-tuning and RAG approaches, a direct comparison to these methods is missing, which is regrettable.", "methods_and_evaluation_criteria": "- AdaptEval covers a broad range of distribution shifts—domain knowledge, instruction tasks, and reasoning—so it appears to be well-suited for evaluating both the generality and performance of the proposed method.", "theoretical_claims": "- The assumption that reducing the model’s input perplexity also lowers its output perplexity is highly convincing, especially given the autoregressive nature of the model and prior findings in language modeling.", "experimental_designs_or_analyses": "Strengths:\n  - By conducting experiments on the AdaptEval benchmark—which spans a variety of tasks—this design is shown to possess strong robustness.\n\nWeaknesses:\n  - Same as the second weakness in \"Claims And Evidence\"", "relation_to_broader_scientific_literature": "- Addressing distribution shifts in deployment environments is a crucial issue in the fields of NLP and deep learning, and this work reflects the latest trends, such as perplexity-based updates and the application of LoRA.\n- AdaptEval appears likely to serve as a unified standard for subsequent research on TTA, TTT, and similar methods.", "essential_references_not_discussed": "- Same as the second weakness in \"Claims And Evidence\"", "other_strengths_and_weaknesses": "Strengths:\n  - Combining high-perplexity sample selection with LoRA to balance computational cost and adaptation performance is conceptually neat and practically beneficial.\n\nWeaknesses:\n  - More detailed comparisons of time complexity and resource requirements would be welcome when handling different types of domains (e.g., conversational tasks, online streaming data).", "other_comments_or_suggestions": "- Overall, this study is well-structured in assuming situations where obtaining high-quality labels is difficult in real operational environments, enabling models to adapt to available test data on the fly.\n- It highlights the importance of test-time learning, particularly perplexity-based updates, in large language models, and underscores the practicality of LoRA for mitigating catastrophic forgetting.\n- If possible, supplementing the paper with more detailed analyses on extreme cases and computational efficiency, along with comparisons to the latest literature, could further bolster its contributions.", "questions_for_authors": "Same as above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lFlU9jAaaV", "reviewer": "Reviewer_LK6b", "summary": "This paper proposes a novel Test-Time Learning (TTL) method that assigns weights to different samples based on input perplexity and employs LoRA for model adaptation. Experimental results demonstrate that, compared with existing TTL methods, the proposed approach achieves superior performance across multiple datasets.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the encouraging comments and detailed suggestions. Responses are below:\n\n>Q1. Regarding the claim of \"Reducing Output Perplexity through Input Perplexity Minimization,\" the theoretical justification provided is somewhat unclear.\n\n**A1.** We appreciate your feedback. We clarify our motivation below:\n* **Theoretical Motivation Clarification.** In our study, we observe the strong positive correlation between input and output PPL (see Fig.1b). Relying on this, we seek to reduce the output PPL by input PPL minimization. \n* **Autoregressive Training Dynamics.** In autoregressive models, each token is generated using the internal representation of the input. Although the model parameters are updated during input PPL minimization, this process refines the representation $h(x;\\Theta)$, which in turn informs the generation of $P(y|x;\\Theta)$. In other words, since $P(y|x; \\Theta)$ is generated directly from $h(x;\\Theta)$, an improved representation of $x$ can lead to more accurate and confident next-token predictions, which is expected to reduce output PPL.\n* **Empirical Validation.** We compare the changes in both input and output PPL between our proposed method and the original LLM. In Tab. A, **our method not only reduces input PPL but also leads to a consistent decrease in output PPL**.\n\nTab. A: Experimental results (metric is PPL) on DomainBench.\n|| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|-|\n|Input PPL|Llama3.1-8B|187.3|933.2|43.6|323.0|\n||Ours|6.6|3.9|2.0|5.5|\n|Output PPL|Llama3.1-8B|3242.9|205247.1|7.7|1208975.9|\n||Ours| 2483.9|156232.3|6.6|242617.0|\n\n>Q2. Need to empirically test various values of $P_0$  on reasoning-oriented tasks. Introducing a more generalizable or theoretically justified criterion for selecting $P_0$ would further strengthen the method.\n\n**A2.** We would like to clarify the applicability of the threshold $\\mathcal{P}_0$ in reasoning-oriented tasks, and outline potential strategies for its dynamic selection:\n* **Generalization of $\\mathcal{P}_0=e^3$ in reasoning-oriented tasks.** We conduct experiments with values of $\\mathcal{P}_0=\\\\{e^2,e^3,e^4\\\\}$. In Tab. B, when $\\mathcal{P}_0=e^3$, our method achieves the best performance on three datasets. These results indicate that $\\mathcal{P}_0=e^3$ generalizes well to reasoning-oriented tasks.\n* **Potential method for dynamic threshold selection.** To address the concern regarding the generalizability of the static threshold, one can adopt a dynamic threshold selection scheme. Instead of a fixed $\\mathcal{P}_0$, we define $\\mathcal{P}_0^{(t)}=\\mu\\_{\\mathcal{P}}^{(t)} + \\alpha \\cdot \\sigma\\_{\\mathcal{P}}^{(t)}$, where $\\mu\\_{\\mathcal{P}}^{(t)}$ and $\\sigma\\_{\\mathcal{P}}^{(t)}$ are the mean and standard deviation of test samples PPL at step $t$. This method automatically identifies high-surprisal samples critical for adaptation while excluding redundant low-PPL data.\n\nTab. B: Effects of different $\\mathcal{P}_0$ under Llama3.1-8B on ReasoningBench.\n|$\\mathcal{P}_0$|$e^2$|$e^3$|$e^4$|\n|-|-|-|-|\n|GSM8K|0.8070|**0.8074**|0.8026|\n|MetaMath|0.7002|**0.7006**|0.7006|\n|Logiqa|0.4834|**0.4868**|0.4538|\n\n>Q3. There is a need to evaluate newer and larger-scale models from multiple model families.\n\n**A3.** We strengthen our empirical validation by expanding experiments across model families and scales. Specifically,\n* **Existing Multi-Family Model Coverage.** As shown in Tables 2-3, our experiments already encompass: **1) Llama Family:** 3B, 8B, and 13B parameter variants. **2) Qwen Family:** Latest Qwen2.5-7B-Instruct model. \n* **New Model Family Added.** To further validate broader applicability, we conduct additional experiments with **Phi-4-14B**. In Tab. C, our method consistently outperforms existing baselines, demonstrating its effectiveness on larger-scale model adaptation tasks. \n\nTab. C: Experimental results on *DomainBench*.\n| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|\n|Phi-4-14B|0.2326|0.0997|0.1291|0.2517|\n|Tent|0.0000|0.1178|0.1262|0.2206|\n|EATA|0.0064|0.0143|0.1388|0.2200|\n|Ours|**0.2421**|**0.1315**|**0.1393**|**0.2711**|\n\n>Q4. Need to add SOTA TTA methods for a more comprehensive evaluation.\n\n**A4.** We agree that evaluating against more recent TTA methods can provide a more comprehensive comparison and strengthen the empirical validation of our method. **We include COME [r1], a recently proposed TTA method accepted at ICLR 2025**, in our experimental comparison. In Tab. D, the results demonstrate that **our method consistently and significantly outperforms COME across all domains**. \n\nTab. D: Comparison between COME and our method on DomainBench.\n| Methods|Geo.|Agri.|Med.|Fin.|\n|-|-|-|-|-|\n|COME [r1]|0.0048|0.0039|0.0301|0.0328|\n|Ours|**0.3212**|**0.1319**|**0.2372**|**0.3242**|\n\n[r1] COME: Test-time Adaption by Conservatively Minimizing Entropy, ICLR 2025.\n****\nWe sincerely hope our clarifications above have addressed your concerns. We would be grateful if you could kindly reconsider the evaluation of our paper.", "claims_and_evidence": "The majority of claims presented in the paper are adequately supported by experimental results. However, regarding the claim of **\"Reducing Output Perplexity through Input Perplexity Minimization,\"** the theoretical justification provided is somewhat unclear. While the autoregressive property indeed implies predicting subsequent tokens based on previous ones, the input content is inherently given (fixed), whereas the output tokens are generated sequentially. Therefore, it is not theoretically evident that minimizing input perplexity directly implies a corresponding reduction in output perplexity.", "methods_and_evaluation_criteria": "Overall, the methods and evaluation criteria employed in this work are reliable and sound. Nevertheless, the choice of the threshold $P_0$ for the perplexity-based weighting scheme warrants further investigation. Currently, the authors have empirically tested various values of  $P_0$  only on DomainBench datasets, without assessing its generalizability to reasoning-oriented tasks. Introducing a more generalizable or theoretically justified criterion for selecting $P_0$ would further strengthen the methodology.", "theoretical_claims": "No theoretical part in this paper.", "experimental_designs_or_analyses": "The experimental design is reasonably thorough and comprehensive. However, the selection of models is relatively limited. Including models such as LLaMA-2-13B, which might be somewhat outdated, is less convincing. It would strengthen the work to evaluate newer and larger-scale models from multiple model families to demonstrate broader applicability.", "relation_to_broader_scientific_literature": "The key contributions of the paper are closely related to the broader scientific literature on large language models and their reasoning capabilities. It addresses the issue of Test-Time Learning.", "essential_references_not_discussed": "There are no essential references missing from the paper.", "other_strengths_and_weaknesses": "See above.", "other_comments_or_suggestions": "See above.", "questions_for_authors": "the baselines used for comparison (Tent and EATA) are somewhat outdated, being methods from 2021 and 2022. Considering recent advances summarized in the survey *\"A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts\"*, are there any newer state-of-the-art test-time learning methods for a more comprehensive evaluation and stronger evidence of the proposed approach’s superiority?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "None"}}, {"review_id": "LVgPPB9nsy", "reviewer": "Reviewer_ftxN", "summary": "The paper introduces Test-Time Learning (TTL) paradigm for Large Language Models (LLMs), termed TLM, designed to dynamically adapt LLMs to target domains using only unlabeled test data during inference. The authors propose three key components: (1) an input perplexity minimization objective, based on empirical and theoretical evidence that reducing input perplexity enhances autoregressive predictions; (2) a Sample Efficient Learning Strategy, which prioritizes high-perplexity samples for efficient model updates; and (3) the use of Low-Rank Adaptation (LoRA) to mitigate catastrophic forgetting and enable lightweight updates. The paper also introduces the AdaptEval benchmark, comprising DomainBench, InstructionBench, and ReasoningBench, to evaluate TTL across diverse tasks. Experimental results claim at least a 20% performance improvement over baseline LLMs on domain knowledge adaptation, with further gains in instruction-following and reasoning tasks, validated on models like Llama3.1-8B-Instruct and Qwen2.5-7B-Instruct.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We are deeply grateful for your thoughtful and encouraging feedback. Your recognition of our motivation and the thoroughness of our experiments is truly inspiring. Our detailed responses are as follows:\n\n>Q1. One experiment question is that how does your methods compared with other adapting methods such as fine tuning and rag.\n\n**A1.** Thank you for the insightful question. Our work focuses on TTL, which adapts LLMs using only *unlabeled test data* during inference. In our experiments, we thus chose to compare our method with other TTA methods, which share a similar problem setting. We would like to **clarify further the rationale for not including fine-tuning and RAG-based methods in our experimental comparison, along with the potential advantages of our method**:\n* **Supervision and Data Dependency.** Fine-tuning requires labeled data, which is costly and often impractical in deployment. RAG depends on a well-maintained retrieval corpus, which may not generalize well across domains. In contrast, **our method uses only test data and minimizes perplexity to adapt efficiently without labels or external knowledge**.\n* **Experimental Design Rationale.** Our experimental design is deliberately aligned with the TTL setting, where no labels or external knowledge sources are available at deployment. Therefore, we select baselines (e.g., Tent, EATA) that also operate in this setting. **Although fine-tuning and RAG are powerful in their respective contexts, they fall outside the scope of our setting due to their dependency on supervised data or retrieval infrastructure.**\n\n>Q2. Reproducibility: Missing details (e.g., random seeds, dataset splits) hinder replication.\n\n**A2.** We appreciate your feedback on ensuring reproducibility. Below we clarify the missing details and emphasize the existing descriptions in our paper:\n* **Random Seeds.** Ours experiments use a fixed random seed of **42** for PyTorch/Numpy initialization.\n* **Dataset Splits for AdaptEval Benchmark.** The proposed AdaptEval benchmark is specifically designed for TTL. Unlike traditional benchmarks, **AdaptEval contains only test-domain data without predefined train/val/test splits**, as TTL methods adapt solely during inference. Full dataset construction protocols are provided in Supp. B.\n* **Additional Reproducibility Guarantees.** **1)** All hyperparameters (learning rates, batch sizes, adaptation steps) are detailed in Sec. 5.1. **2)** Hardware specifications (e.g., GPU types) and library versions (PyTorch) are detailed in Supp. C.2. **3)** Code, pre-trained models, and AdaptEval data will be publicly released upon acceptance.\n\n\n>Q3. Consider moving supplementary metrics (e.g., BERTScore) to the main paper for a richer evaluation.\n\n**A3.** We appreciate the feedback and agree that incorporating supplementary metrics like BERTScore into the main paper will strengthen the evaluation's comprehensiveness. \n\n>Q4. Expand the discussion of trade-offs in the online setting (e.g., adaptation quality vs. efficiency).\n\n**A4.** Thanks for your valuable comments. We conduct additional experiments in the **online setting** to evaluate the effect of batch size under streaming-like scenarios (see Table C). **With bs=1, the model updates after each test sample, simulating a true streaming setting. While this increases update steps, our method remains effective.** Using bs=100 reduces update frequency while slightly improving performance. This indicates that accumulating more samples before updating provides more reliable learning signals, and thus enhances the adaptation performance. The results suggest that in an online setting, adopting as high a batch size as possible can lead to both effective and efficient performance. The results suggest that in the online scenario, while more frequent updates offer immediate adaptation, adopting a larger batch size can lead to more effective and efficient adaptation.\n\nTable C. Experimental results in the **Online setting** on DomainBench.\n| Methods|Average R-Lsum|#Backwards|\n|-|-|-|\n|Ours (bs=100)|0.2040|1514|\n|Ours (bs=1)|0.1917|2541|\n\n>Q5. I did not quite understand the Eq 4, why Minimizing the perplexity to the input $\\mathcal{P}(x;\\Theta)$ is equivalent to maximizing the input generation probability $P(x;\\Theta)$. \n\n**A5.** Thank you for your insightful suggestion. The equivalence between minimizing input perplexity and maximizing the input generation probability is a fundamental property of perplexity in probabilistic models. We clarify this relationship as follows. Perplexity $\\mathcal{P}(x;\\Theta)$ for a sequence $x$ is defined as the exponential of the **cross-entropy** loss $P(x;\\Theta)$, Formally, Eq. (2) can be written as：$\\mathcal{P}(x;\\Theta)=e^{-\\frac{1}{N}\\log P(x;\\Theta)}$, where $N$ is the number of tokens in the sequence. Therefore, maximizing $\\log P(x;\\Theta)$ directly corresponds to minimizing $\\mathcal{P}(x;\\Theta)$.\n****\nWe sincerely hope our clarifications above have addressed your concerns.", "claims_and_evidence": "The claims are generally supported by clear and convincing evidence, including empirical observations, theoretical justifications, and experimental results. \n\nThe core claim—that minimizing input perplexity improves LLM performance—is backed by Observation 1 (Fig. 1b)\n\nThe claim that high-perplexity samples are more informative is supported by Observation 2 (Fig. 1c)\n\nExperimental results (Tables 2, 3, 4) consistently show TLM outperforming baselines (e.g., Tent, EATA) and original LLMs, with a reported 20%+ improvement on DomainBench", "methods_and_evaluation_criteria": "The proposed TLM method—combining perplexity minimization, sample-efficient learning, and LoRA—makes sense for adapting LLMs to distribution shifts without labeled data, aligning with the problem of real-world deployment challenges. \n\nThe use of perplexity as an unsupervised objective leverages LLMs’ autoregressive nature, which is a conceptually sound departure from entropy-based TTA methods ill-suited for LLMs.", "theoretical_claims": "I checked the correctness of the theoretical justification for input perplexity minimization (Sec. 4.1, Eqns. 4-7).\n\nThe argument has on two assumptions: (1) the autoregressive property of LLMs and (2) shared parameter influence on input and output perplexity.", "experimental_designs_or_analyses": "I reviewed the experimental designs in Sec. 5 and Supp. D. The comparison experiments (Tables 2, 3) against baselines (Tent, EATA) and original LLMs are well-designed, using diverse models and datasets to test robustness.\n\nOne experiment question is that how does your methods compared with other adapting methods such as fine tuning and rag.", "relation_to_broader_scientific_literature": "The paper builds on prior work in test-time methods for classification. The perplexity minimization objective extends autoregressive modeling principles, shifting them to test-time adaptation.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "strength:\n1. The paper is well-structured, with clear figures (e.g., Fig. 1) and a logical flow from theory to experiments.\n2. The focus on unlabeled test-time adaptation and the AdaptEval benchmark tackle real-world LLM limitations.\n\nweakness: \n1. Reproducibility: Missing details (e.g., random seeds, dataset splits) hinder replication.", "other_comments_or_suggestions": "1. Consider moving supplementary metrics (e.g., BERTScore) to the main paper for a richer evaluation.\n\n2. Expand the discussion of trade-offs in the online setting (e.g., adaptation quality vs. efficiency).", "questions_for_authors": "I did not quite understand the Eq 4, why Minimizing the perplexity to the input P(x; Θ) is equivalent to maximizing the input generation probability P(x; Θ).", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Test-Time Learning (TTL) for large language models (LLMs), a method that adapts LLMs to domain shifts using only unlabeled test data. The key idea is to minimize input perplexity (PPL) rather than output entropy, as a self-supervised objective, leveraging the observation that high-PPL inputs are more informative for adaptation. The authors propose a sample-efficient strategy that prioritizes such inputs and employs LoRA for stable updates, avoiding catastrophic forgetting. They also introduce AdaptEval, a benchmark covering domain-specific, instruction-following, and reasoning tasks. Experiments show consistent improvements over baselines, demonstrating the method’s effectiveness in real-world scenarios where labeled data is scarce.

Reviewers acknowledged the practical contributions but raised concerns: (1) Theoretical gaps in linking input PPL minimization to improved output predictions; (2) Unclear novelty compared to prior TTA and perplexity-based methods; (3) Limited comparisons with recent TTA techniques (e.g., COME) and alternative approaches like fine-tuning/RAG;

After the rebuttal, the authors addressed concerns by: (1) Adding gradient-based theoretical analysis to justify the input-output PPL connection, supported by empirical validation; (2) Expanding experiments to include comparisons with COME (ICLR’25) and evaluations on longer-response tasks; (3) Clarifying distinctions between TTL and TTA. However, some concerns remain. The theoretical claims still lack rigor, the novelty of PPL minimization versus entropy minimization needs deeper ablation, and the comparison with TTA methods needs to be included.

Most of the reviewers agreed that some key concerns have been addressed, while the presentation needs substantial modification at current stage. Given these considerations, I recommend weak acceptance. The authors should improve the paper by incorporating all the discussions and feedback during the rebuttal.
Decision: Accept (poster)