Unified Transferability Metrics for Time Series Foundation Models
Authors: Weiyang Zhang
Xinyang Chen
Xiucheng Li
Kehai Chen
Weili Guan
Liqiang Nie
Venue: NeurIPS 2025 poster ()
Keywords: Time Series Analysis; Transferability Estimation; Pre-trained Model
Primary area: deep_learning

Abstract: With the increasing number of time series pre-trained models, designing transferability evaluation metrics for time series has become an urgent problem to address. 
While transferability evaluation has been extensively studied in computer vision, we aim to address a critical gap by developing tailored metrics for time series analysis. 
In this paper, we introduce TEMPLATE, a transferability estimation framework specifically tailored for versatile time series analysis, comprising three complementary metrics: (1) Dependency Learning Score quantifies a model’s capacity to capture temporal dependencies. (2) Pattern Learning Score evaluates the representation quality in extracting discriminative temporal patterns. (3) Task Adaptation Score assesses cross-task generalization capability, enabling versatile time series analysis. TEMPLATE presents a versatile framework compatible with both classification and regression paradigms. Through comprehensive benchmarking across 5 distinct downstream tasks, our method demonstrates superior capability in identifying optimal pre-trained models from heterogeneous model pools for transfer learning. Compared to the state-of-the-art method ETran, our approach improves the weighted Kendall's $\tau_w$ across 5 downstream tasks by 35\%. The code is available at https://github.com/ooooooover/TEMPLATE.

Unified Transferability Metries for Time Series Foundation Models
Weiyang Zhang' Xinyang Chen ls
Xiucheng Li' Kehai Chen'
Weili Guan Liqiang Nie'
School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen)
2 School of Information Science and Technology, Harbin Institute of Technology (Shenzhen)
(zhangweiyang12233. chenxinyang95, nieliqiang) aggmai.. aeggail..omm com (lixiucheng, chenkehai, guanweili] @hit. edu. cn
Abstract
With the increasing number of time series pre-trained models, designing transfer- ability evaluation metrics for time series has become an urgent problem to address. While transferability evaluation has been extensively studied in computer vision, we aim to address a critical gap by developing tailored metrics for time series analysis. In this paper, we introduce TEMPLATE. a transferability estimation framework specifically tailored for versatile time series analysis, comprising three complementary metrics: (1) Dependency Learning Score quantifies a model's ca- pacity to capture temporal dependencies.
(2) Pattern Learning Score evaluates the representation quality in extracting discriminative temporal patterns. (3) Task Adap- tation Score assesses cross-task generalization capability, enabling versatile time series analysis. TEMPLATE presents versatile framework compatible with both classification and regression paradigms. Through comprehensive benchmarking across 5 distinct downstream tasks, our method demonstrates superior capability in identifying optimal pre- rreitrinedd models from heterogeneous model pools for transfer learning. Compared to the state-of-the-art method ETran, our approach improves the weighted Kendall's Tw across downstream tasks by 35%. The code is available at https://gituub.com http:iiiuwbooom/EEMPMP.P.. TEMPLATE
Instroduction
Recently, pre-trained models have drawn increasing attention in the time series domain due their exceptional performance in computer vision and natural language processing [ These models
have achieved significant success across various time series downstream tasks (2] and are readily available on platforms like HuggingFace 4 and TensorFlow Hub [5]. However, no single model consistently outperforms others across all datasets. Therefore, selecting the most suitable pre-trained time series model for a given target task has become a pressing challenge. A time-consuming solution fine-tune all pre-trained models on the target dataset and then select the best-performing fine-tuned model. But compared to pre-traine trained models in computer vision such as ResNet [6) and MobileNet 77, time-series pre-trained models have much larger parameter scales g making direct brute-force fine-tuning incur enormous time costs and high computational resource requirements 100, as shown in the left part of Figure
Recent studies propose fast transferability evaluation methods to efficiently rank models and select the optimal one. Existing methods can generally be categorized into static and dynamic approaches M. Static methods calculate scores directly based on the statistical information of the model, such as LEEP [12, NLEEP [3], H-score [14 and TMI [15. In contrast, dynamic methods transform this statistical information using certain learning frameworks or representation space mapping algorithms before calculating scores, such SFDA [16]. LogME [17], and ETran [18) These approaches are empirically validated as effective metrics for selecting computer vision models.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Porquasing
Model Pool
Accurate Time-consuuinng
Model
DCX
Impetation
Atemay Oetrction
Fine-tuning
Metrics IAcc. MSE, F2 s..oooe
MONENT
Propecties time series analysis (1) Time series diverse tasiss
(2) Large Architectures.
Heterogeneous
TIMASAA CO
(1) Gross task generalization bottleneck (2) High-dimensional infeacibility.
(3) Peatuare space mismatch.
Designed Metric Accurate and Eilictent
Fine-ture (h)
Figure I: Pre-trained models' fine-tune overhead (left) and transferability metric framework (right).
Nevertheless, existing research primarily focuses on image-based tasks. When applied to time series, several challenges emerge, shown in the right part of Figure D First, mainstream time series tasks exhibit significant diversity, encompassing forecasting, classification, imputation, and anomaly detection, among others. While existing methods predominantly concentrate classification- based tasks, achieving cross-task generalization capability remains the primary challenge. Second, downstream time series datasets frequently exhibit large-scale characteristics, resulting in high- dimensional computational infeasibility. This consequently imposes stringent requirements on evaluation metrics, demanding both computational efficiency and measurement accuracy. Last but not least, computer vision models consistently employ standardized data preprocessing protocols, consequently maintaining minimal variance in feature dimensionality, But in time series analysis, the architectural heterogeneity of models and their divergent approaches to processing inter-channel dependencies lead to substantial discrepancies in feature dimensionality. This consequently induces feature space mismatch, rendering evaluation metrics that rely on numerical magnitudes such as intra-class variance [9] or singular value [17] ineffective. Table Soole[[ssma summarizes the challenges in developing transfcrability metrics for time series. Prior to this paper, there is no ideal solution for pre-trained model selection specially designed to tackle these challenges.
To address these fundamental challenges while accounting for the intrinsic characteristics of time series, we propose TEMPLATE (TEMPoraL representAtion Transferability Estimation), which quantitatively assesses the transferability of time series pre-trained models through three comple- mentary evaluation metrics: Firstly, effective time series pre-trained model must demonstrate robust capability in capturing and representing temporal dependencies. Moreover, we empirically demonstrate that the principal components constitute the most transferable elements within learned representations. Consequently, we propose the Dependency Learning Score as a quantitative metric to assess dependency learning capability through comparative analysis of principal components between original representations and their corresponding representations in trend. Secondly, to assess the core ability of time series pre-traine pre-trained models in capturing key temporal patterns, we introduce the Pattern Learning Score as quantitative evaluation metric. This score precisely measures the model's focus on key temporal patterns through the ratio of the principal component to the overall pattern complexity. Finally, to ensure adaptability across diverse downstream tasks and enable versatile time series analysis, we introduce the Task Adaptation Score as task-specific evaluation metric for assessing representation quality. For high-level tasks, hierarchical representations are preferred. Thus we calculate the similarity between the features output by the first and last layers of the encoder. Through the integration of these three complementary evaluation metrics, establish comprehen- sive framework for assessing time series pre-trained models. Experimental results demonstrate strong correlation between proposed metric and actual model transferability, consistently outperforming existing evaluation methods in benchmark tests across 5 downstream tasks.
Our contributions are summarized as follows:
We evaluated the challenges in designing transferability metrics for time series and, from the model's dependency learning ability, pattern learning ability, and task adaptation ability, proposed the TEMPLATE pre-trained model transferability evaluation framework.

---

Table 1: Time series design challenges and methods comparison.
Challenges
NCE LEEP
LogME SFDA NLEEP PACTran TMI ETran NCTI Ours
Cross-task generalization bottleneck
X
X
K
K
X
K
d
X
High-dimensiioal infeasibility
K
X
Feature space mismatch
X x
x
X
X
8
x
x
Our proposed framework is highly flexible and generalizable, supporting both classification and regression tasks. Moreover, the metrics can he applied to 5 mainstream time series analysis tasks, demonstrating our broad applicability and task-specific adaptability.
Experiments on 32 datasets across 5 mainstream time-series tasks demonstrate that our method achieves state-of-the-art (SOTA) performance in evaluating the transferability of time-series pre- trained models, with an average improvement of 35% compared to previous SOTA methods.
2 Related Work
2.1 Time Series Pre-trained Model
In the image and vision domains, pre-training large-scale sequence datasets has significantly advanced modality understanding [1]. Building on this, pre-trained models have been widely
developed in the time series domain 11. LLM4TS [22 enhances forecasting accuracy by fine-tuning
large language models, while Tempo 23 improves cross-modal alignment using multimodal prompts.
Lag-Llama [24]] conducts pre-training on multi-domain time series data and uses lag features as covariates for probabilistic univariate forecasting. Moment (25] conducts masked reconstruction
pre-training on various time series datasets. Similarly, Timer undergoes autoregressive pre-training
on scale collected datasets. UniTS [26] integrates downstream tasks into a single framework through task tokenization. Time-MOE (27, which combines the mixture-of-expprts method, the first to scale time series pre-trained models to 2.4 billion parameters. Therefore, as the scale of time-series pre-trained models continues to grow, how evaluate the transferability of these models has become an urgent issue that needs to be addressed.
2.2 Model Transferability Estimation
Model Transferability Estimation (MTE) aims to provide fast and efficient methods for quantifying model's performance on downstream tasks without fine-tuning [11. Recent MTE advancements
focus on vision models. Early works, such as NCE 28 and LEEP 12.. focused on probabilistic methods based the expected empirical distribution of target labels. LogME [17] estimates the maximum marginalized likelihood of labels, while NLEEP [3 extends LEEP by replacing the output layer with Gaussian Mixture Model. SFDA [6 projects features into Fisher space and enhances class separability using mhysics-inspired models. TMI [15] uses intra-class feature variance performance indicator, assuming that lower variance reflects tighter clustering of class features. NCTI [1), inspired by the neural collapse phenomenon, develops metrics to measure the distance between the current state of the pre-trained model and its hypothetical fine-tuned state. ETran [18] introduces energy scores to quantify whether the target dataset is in-distribution or out-of-distributiin for candidate model, assuming models with higher in-distribution levels for the target dataset exhibit greater transferability. Existing studies primarily focus on vision and have not fully considered the characteristics of time-series data and pre-trained models. Therefore, propose TEMPLATE, transferability evaluation framework specifically designed for time-series pre-trained models.
3
A Approach
3.1
Problem Formulation
Given M pretrained models {dm )MM and target dataset D with N samples, where each model consists of a backbone / that outputs encoded features, and the target dataset associated with an evaluation metric (accuracy, MSE, F-score etc.), we initialize predictor head on the backbone / and fine-tune the entire model on the target dataset. The feature extracted by the I-th layer of the pretrained model (..()) (-) is denoted as H where H' om (X) Rg. and d is the feature dimension. In the

---

following, we omit i using H to denote the feature output of the last layer of the model's 's encoder. By performing brute-force fine-tuning on all models, we obtain the true performance {Pm JM m= for the model hub. A practical model selection method should generate score for each pretrained model. Ideally, the score should correlate strongly with the actual fine-tuning performance {Pm m=11 allowing the best- best-performing pretrained model to be selected based on the evaluation score alone.
3.2 Transferability Assessment through Temporal Representation Transferability Estimation
To effectively evaluate the transferability of time series pre-trained models, it is necessary to address the dimensional gap in feature matrices caused by the model's different handling of inter-channel relationships, well the varying requirements of various downstream tasks. Therefore, we avoid directly using statistical quantities of the feature matrix to measure the transferability of pre- trained models. Specifically, we propose TEMPLATE, a hybrid transferability metric that includes Dependency Learning Score, Pattern Learning Score, and Task Adaptation Score, aiming to assess transferability of the learned representation in pre- pre-trained models from the perspective of temporal dependency learning, temporal pattern learning and hierarchical features.
To better understand how fine-tuning affects the representation learned by pre-trained models, explore the changes in the components of feature matrices before and after fine-tuning. Specifically, use Singular Value Decompo- sition (SVD) to analyze this process as it ef- ficiently decomposes the feature matrix and analyzes its dominant patterns. We fine-tune MOMENT-small [25). Timer-small ( and UniTS-small 266 on EthanolConnentrratoon [29].
similarity
similarity
Cosine similarty
MENESTO
costre sin sinfarty
codre simiartty
Handwriting [29] and UWaveGestureL
(99 dataset, respectively, and compare the co- sine similarity of the feature vectors correspond- ing to singular values before and after fine- tuning. The results are shown in Figure and is observed that the cosine similarity of feature vectors decreases as their corresponding singular values decrease. This indicates that larger singu- lar values encapsulate more knowledge and are more transferable to downstream tasks.
Index
Indcx
Indcx
Figure 2: The cosine similarity of feature vectors, computed based on feature vectors before and after fine-tuning, on three datasets (sorted in descending order by singular values): EchanolConnenttration (Left), Handwriting (Middle), and UWaveGesture- Library (Right).
Dependency Learning Score. The core of time series modeling is to capture temporal dependency. Time series pre-trained models should be able to capture long-term dependencies, which represent stable and persistent patterns crucial for effective modeling [30]. To assess this ability, we employ series decomposition 31 to extract the trend component, which inherently contains long-term dependencies. This component fed into the model, yielding a feature matrix T that is focused on capturing long-term dependencies, as follows:
(I) where we use padding to maintain the original series length, and then apply the AvgPool layer for moving average calculations. A model proficient in learning long-term dependencies should exhibit high degree of consistency between the feature matrix H and the feature matrix T of the trend component. To this end, we propose the Dependency Learning Score Sa. Specifically, considering that the largest singular value and its corresponding feature vector demonstrate superior transferability. we prioritize the use of the feature vectors corresponding to the largest singular values of H and T. We perform SVD to decompose the features:
(x) AvgPool(ppaddinn (X)nddinnng()).nn...)) T = &m (trend(X)),
H = U. VT. T V.
We denote the largest singular values of H and T as σn and o, and their corresponding feature vectors as uy and ue, respectively. With the feature vectors, Sau is formulated to quantify the pre-trained model's capability in capturing long-term temporal dependencies through computation of the Pearson correlation coefficient 332 between the un and ue, as follows:
Sai p(un, w)) = Cov(up: u.) An
4

---

where Cov(.) represents the covariance between feature vectors, and A, A represents their respective variances. Since Sa represents the pre-trained model's ability to learn long-term dependencies. a pre-trained model that effectively learns long-term dependencies should achieve a larger Sat
Pattern Learning Score. Effective temporal pattern learning is critical for time series pre-trained models to handle complex, multi-scale behaviors, which allows the model to learn the temporal dynamics in time series data, thereby achieving superior performance on downstream tasks.
To this end, we propose the Pattern Learning Score Sp to measure the pre-trained model's ability to learn the primary temporal patterns. Within the framework of matrix decomposition, singular values quantitatively characterize the relative significance of distinct patterns embedded in the feature matrix. The largest singular value typically corresponds to the primary temporal pattern in time series, reflecting the most significant dynamic behavior captured by the model.
While the dominant singular value provides an intuitive measure of a model's 's capacity capture primary temporal patterns, the inherent feature space mismatch resulting from heterogeneous inter- channel relationship processing across pre-trained models may significantly compromise the reliability of transferability assessment. Therefore, we introduce the nuclear norm as a measure of the overall importance of the patterns, defined as the sum of all singular values of the matrix. Specifically, we use the ratio of the largest singular value to the nuclear norm to measure the pre-trained model's ability to learn primary temporal patterns. A higher ratio signifies enhanced capability of the pre- trained model in capturing dominant temporal patterns, whereas lower ratio potentially indicates insufficient representation of these critical temporal patterns. Moreover, compared to the raw data, the temporal patterns of the trend component are simpler, with the primary patterns being more prominent. Therefore, the ratio of the largest singular value to the nuclear norm using the trend component provides more accurate evaluation of the pre-trained model's ability to learn primary temporal patterns. The formulation of Spt is as follows:
Sp = O2
where of denotes the largest singular value of the trend component feature matrix, and ||T|| represents its nuclear norm. Since Spt represents the pre-trained model's 's ability to learn primary temporal patterns, a pre-trained model that effectively learns these patterns should achieve a larger Spt.
Task Adaptation Score. Time series downstream tasks exhibit substantial heterogeneity, with dis- tinct tasks imposing markedly divergent requirements on feature representations [33]. Classification tasks require high-level semantics, and imputation tasks require the model to learn from missing data. Consequently, hierarchical representations become essential. In contrast, forecasting tasks and anomaly detection tasks place more emphasis on the precise extraction of low-level features from the raw data itself, focusing on local patterns and temporal dependencies an effectively designed pre-trained model should demonstrate the capability to learn task representations that dynamically adjust to varying downstream task requirements. [35). Moreover, due to the diverse designs of tine-seriss pre- pre-trained models and the significant differences in network architectures. it is necessary adaptation ability of the pretrained model by calculating the similarity between the output features H from the first layer of the encoder and the output features H from the last layer. A simple approach is to directly compute the dot prduuct-based similarity (36] between them:
35 33]. Consequently,
consider the overall evolution of features. We measure the task
(Pec(H) (H) v)c(HH())) = ur(H (H') = I
Due to the differences in how time series pre-trained models handle inter-channel relationships, there is significant gap in feature matrix dimensions across different pre-trained models. Directly calculating squared dot product can lead to inaccurate estimates of model's transferabilitty Therefore,
we use centered kernel alignment [37 38 36 to measure similarity, which further optimizes dot
product-based similarity. Specifically, Equation 5 means that, for the centered H' and H:
(n 1)2 tr (H") = [Cov((H))) H )1.
1
(6)
The Sehheet-SSchiidt Independence Criterion (39
generalizes Equations 5 and inner products from reproducing kernel Hilbert spaces, where the squared Frobenius norm of the cross-covariance matrix
5

---

becomes the squared Hillert-Schhice norm of the cross- covarrince operator. Let Kg = k(x,,,)) and Lij- (y))),)) where & and I are two kernels. The empirical estimator of HSIC is:
HSIC(K, (n - 1)² tr QQ)))
(7)
where Q is the centering matrix Qn = 1n - 111 -11 Finally, we normalize the HSIC to obtain our score function, Sya:
HSIC(K, L)
Sta
(8)
V HHSICKK, K ()HSIC(L, L)
where the RBF kernel xx) exp(-l|x; x211B/2>?)) for mapping, with several possible strategies for selecting the bandwidth. This strategy controls the extent to which similarity at small distances emphasized over large distances. We 7 to the median distance between samples. For classification and imputation task, it is essential for the model to learn hierarchical representations, making lower Sta more desirable. In contrast, for forecasting and anomaly detection task, the model needs to focus on capturing fundamental features, making higher Sta more appropriate.
Overall Score Function. In this paper, expect pre- traitranned models learn valuable temporal dependencies and patterns from the time series data and output task adaptive ability tailored to different downstream tasks, enabling versatile time series analysis. To this end, design three score functions: Sa, Spt. and Sta- Sa and Spt measure the dependency learning ability of the model and primary temporal pattern learning, respectively. S.. evaluates the task-adaptive ability of the pre-trained model to various downstream tasks. Since time series often involves large-scale datasets, directly applying SVD can result in high computational costs. To accelerate the computation, the power iteration method [40] [4] to approximate the largest singular value of the model and its corresponding eigenvector. For detailed information about the power iteration method, please refer to Appendix C.4. Secondly, we quickly calculate the nuclear norm using equation 9 as shown below:
use
|T||- = tr (VTTTT
By employing these two methods, we avoid the high computational cost associated with SVD, enabling fast evaluation of the transferability of pre-trained models. In addition, since the score functions have different scales, directly summing these scores may negatively impact the performance of transferability evaluation. Therefore, we normalize each score to a unit range (0 to 1) instead of manually defining hyperparameters to balance the contributions of each score, as follows:
Sa min(Sa)
Sa (Sa) min(Sa) (Sa)
(10)
Similarly, we can obtain the normalized scores (T) and (HH H). By combining these three scores, we derive the final transferability evaluation metric:
Stotal Sa (H, T) + (T) ASS. (H, H),
(11)
where À is determined by the downstream task type. When evaluating on classification task and imputation task, is set to -1, For forecasting task and anomaly detection task, À is set 1. Note that the three scores are equally weighted, and À acts as a sign function determined by the task type, without requiring fine-tuning for specific downstream dataset. The pre-trained model with higher overall score Stotal indicates better transferability within the model pool for the target dataset D.
Experiments
In this section, examine the effectiveness of TEMPLATE in assessing the transfer learning performance of pretrained models across 5 mainstream downstream time series tasks, including classification, short- and long-term forecasting, imputation, and anomaly detection.
Implementation Details. To quantify the correlation between estimated assessment scores and actual fine-tuning results, we employ the weighted Kendall's T., (see Appendix C.3) which measures ranking agreement with higher weights for higher ranks. More details about the dataset and experiment implementation can be found in Appendix C

---

Table 2: Comparison of SOTA Rate and Relative Improvement for Different Methods.
Method
GBC
42
CC-FV
43
NCTI [9]
ETran [8
Ours
SOTA Rate in Original Paper
37%
36%
40%
45%
46 %
Improvement (vs Previous SOTA)
20%
9%
21%
18%
35 %
Pre-trained Models. We select 10 pre-trained models from 4 model families to form the pre-trained
model pool, including MOMENT-small [25]. MOMENT-base [25], MOMENT-large 225]. Timer-
small (9). Timer-base 19, TimerXL-small 44 TimerXL-base [44, UniTS-small [26], UniTSS base
(26), and --irge (66.. We fine-tune all source models on the target dataset to obtain the actual
ranking. More details about the pre-trained model pool can be found in the Appendix
4.1 Main Results.
Cassiication
TIMR KES
TH822220
As a transferability evaluation metric designed for pre- trained models in time series, TEMPLATE achieves con- sistently ffat-oof-the-ar accuracy across mainstream analytical tasks, as shown in Figure Notably, no method has achieved SOTA all dataset, and illustrated in Table 2 our method attains a remarkably high SOTA rate. We show more experimental results in Appendix [ to further demonstrate the effectiveness of the method.
LegMSD0221
Shork-arm Forcaating
Anomaty Detection
4.2 Classification
Figure 3: Performance comparison.
Setup. Time series classification can be applied tasks such recognition and medical diagnosis [45] Specifically, We verify all methods on 9 multivariate datasets from the UEA classification archive [29], covering gesture, audio recognition, etc., across multiple task categories. We preprocess the datasets according to 46). where different subsets have varying sequence lengths.
Table 3: Classification Benchmark: Performance (Weighted Kendall's 's Tw)) of different methods. The best results are highlighted in bold while the second best underlined.
LogME NLEEP (2021)
SFDA PACTran
TMI (2023)
NCTI ETran RetMMD (2023)
TEMPLATE
Datasets
(2022)
(2022)
(2022)
(2023)
(2024)
(Ours)
EunanolConcentratioon
0.567
0.432
0.120
0.488
-0.430
-0.32
0.686
0.512
0.724
FaceDetection
-0.203
0.092
0.598
0.306
-0.600
0.109
-0.359
0.310
0.597
Handwriting
-0.445
-0.104
0.314
0.596
0.768
0.700
0.478
0.365
0.822
Japanese Vowels
0.231
0.213
0.021
0.306
0.302
0.340
-0.196
0.654
0.447
PEMS-SF
-0.472
-0,612
-0.312
0.306
-0.300
0.053
0.076
0.520
0.470
SelfRcgulationSCCP1
0.356
0.529
0.459
0.619
0.300
0.310
0.651
0.450
0.484
SelfRegulationSCCP2
0.268
0.241
-0.198
0.457
0.455
0.450
0.667
0.397
0.551
SpokenArabicDigits
0.342
0.321
-0.367
0.744
0.647
-0,210
0.479
0.201
0.637
LWareGestureLLbbraayy
0.584
0.127
0.440
0.592
0.576
0.245
0.624
0.362
0.719
Average
0.136
0.138
0.119
0.490
0.191
0.186 0.345
0.419
0.608
Performance Comparison. We compare our method with existing transfcrability evaluation met- rics, including LogME [7], NLEEP [33, SFDA [16], PACTran [47]. TMI [15), NCTI [19]. ETran [8 and RetMMD 48 Table shows the results of our method compared to previous work on classification benchmarks. Our method outperforms all previous methods, achieving a state-of-the-ar (SOTA) Tur of 0.608, which is 24% ahead of the second-bes -bett method. Due to the different ways models handle inter-channel relationships, feature-statistics based methods, such as LogME [17], TMI [15), and NCTI 19). generally perform poorly on datasets with a large number of channels, like FaceDetection and PEMS-SF. In contrast, our method achieves outstanding results on these two datasets, with Tu of 0.597 and 0.470, respectively. These results highlight the effectiveness of our approach in evaluating pre-trained time series models for transfer learning.
7

---

4.3 Short- and Long-term Forecasting
Time series forecasting plays a crucial role in areas such as weather forecasting, traffic management, and energy consumption. To thoroughly evaluate the generalizability of our approach in forecasting, we employ two types of benchmarks: long-term forecasting and short-term forecasting. For long-term forecasting, we use seven widely recognized long-term time series forecasting datasets 31 covering practical applications across various domains. For short-term forecasting, we adopt the M4 dataset [49], which includes univariate marketing data collected annually, quarterly, and monthly,
Setup.
Table 4: Long-term Forecasting Benchmark: All the results are averaged from 4 different prediction
lengths. '-' means out of memory. See Table 12 in Appendix for the full results.
Methods
ETTh1
ETTh2
ETTml
ETTm2
Weather
Electricity
Traffic
Average
LogME
0,215
0.167
0.400
0.565
0.114
-0.130
0.190
ETran 0.138
0.212
0.189
0.351
0.474
0.302
0.192
0.265
Ours
0.240
-0.003
0.518
0.576
0.412
0.361
0.432
0.362
Performance Comparison. Tables 4 and present the experimental results of our method in long-term and short-term forecasting scenarios. We compare our method with existing transferability
evaluation metrics for regression tasks, including LogME [17] and ETran [8] The core of long-term forecasting task lies in the model's s ability to effectively capture complex temporal dependencies and long-term trend. By contrast, short-term forecasting task focus more on the precise capture of fine-graines temporal patterns. Overall, our method demonstrates outstanding performance across both scenarios. Notably, LogME performs poorly on multi-channel datasets like Electricity and cannot evaluate the performance of pre-trained models Traffic due out of memory issues. In contrast, our method achieves excellent results on both datasets. These results further validate the effectiveness of our method in assessing time series pre-trained models for transfer learning.
4.4 Imputation
Setup. Real-time systems are continuously operating and monitored by automated observation devices. However, due to failures, the collected time series may be partially missing, makes down- stream analysis challenging. Therefore, imputation has become widely used in practical applications. In this paper, we select datasets from electricity, weather, and traffic scenarios as benchmarks.
Table 5: Imputation Benchmark: All results are averaged from 4 mask ratios. +_' means out of memory. See Table 13
Appendix for the full results.
Methods
ETTh1
ETTh2
ETTml
ETTm2
Weather
Electricity
Traffic
Average
LogME
0.322
0.238
0.607
0.388
0.312
-0.061
0.258
ETran
0.324
0.419
0.254
0.310
0.441
0.456
0.398
0.372
Ours
0.406
0.317
0.664
0.650
0.645
0.412
0.529
0.518
Performance Comparison. Effective time series imputation places high demands on the model's s ability capture temporal dependencies and understand contextual information. Specifically, when imputing missing values, the model needs to utilize the implicit temporal patterns within data, including modeling dynamic relationships between time points and accurately inferring both the overall trends and local features of time series. Table presents the experimental results of our method in the time series imputation scenario. In the experiments, we evaluate the model's performance under different missing ratios. The results show that, whether in scenarios with low or high proportion of missing data, our method demonstrates high level of model transferability evaluation accuracy.
4.5 Anomaly Detection
Anomaly detection in sensor data crucial for industrial maintenance, as anomalies are often hidden within large-scale datasets, making data labeling challenging. We compare 5 widely used anomaly detection benchmarks: SMD [50). MSL. [51] SMAP [5T], SWaT [52]. and PSM [53] covering applications in service monitoring, spatial data, earth exploration, and water treatment.
Sctup.
8

---

Table Anomaly Detection Benchmark: Performance (Weighted Kendall's Tw) of different methods. The best results are highlighted in bold and means out of memory.
Methods
PSM
MSL
SMAP
SMD
SWaT
Average
LogME
0.230
0,468
0.340
0.207
ETran
0.433
-0.024
0.121
0.126
0.277
0.187
Ours 0.142
0,515
0.431
-0.08
0.173
0.236
Performance Comparison. Effective time series anomaly detection requires the model to identify normal and abnormal patterns in the data and accurately distinguish between the two. Table the results of our method in the time series anomaly detection scenario. The experimental results show that method effectively evaluates the model's transferability in anomaly detection scenarios.
0.6
Methods
LogME
NLEEP
0.5
SFDA
PAC
0.4
NCTI
0.3
ours
Sa
Sp
Se
winde
So
Se
0.2
0.1
Figure 4: Effectiveness of component in TEM- PLATE Classification and Imputation task.
10a
Running Time (5)
10a
Figure 5: Time complexity analysis.
4.6 Analysis
Ablation Study. Our TEMPLATE consists of three sub-items designed to evaluate the performance of pre-trained models downstream tasks. To explore the individual contribution of each sub-item to the final metric and analyze its impact on overall performance, we conduct an ablation study. Figure presents the results of the ablation study on the classification and imputation task, with the complete results available in Appendix D.2 It can be observed that all three metrics achieve positive ranking correlations, and their combination yields the highest average ranking correlation, indicating that the three metrics complement each other and can comprehensively measure the performance of pre-trained models.
Time Complexity. In the Figure 5 we present a comparison of the average runtime and Tu on the classification task across datasets using different model selection metrics. While NCTI [9] and TMI [15] are fast, they perform poorly in evaluating the transfcrability of time series pre-trained models due their reliance statistical metrics feature matrices. LogME [17] is efficient, but cannot be applied to large datasets like Traffic and SMD because it relies on the magnitude of singular values. In contrast, our method uses the power iteration method avoid high computational costs and enabling its application to large datasets. Although ETran [18] efficiently obtains the Sen score, the LDA-based score calculation incurs significant time costs, resulting in slower overall speed.
Conclusion
In this paper, we propose TEMPLATE, a pre-trained model evaluation framework specifically designed for time series. We are the first design transferability evaluation metric tailored for time series. TEMPLATE consists of three scores, aimed at measuring the transferability of pre-trained models from two perspectives: dependency and pattern learning, and task adaptation. Our approach has been successfully applied to 5 mainstream time series downstream tasks. Extensive experiments demonstrate that our method achieves state-of-the-art (SOTA) performance evaluating time series pre-trained models for transfer learning.

---

Acknowledgements
This work was supported by the National Natural Science Foundation of China (62306085, 62206074, 62476071, U23B2055, U24A20328), Shenzhen College Stability Support Plan 1XXWD2023113011122222
CCF-ALIMAMA TECH Kangaroo Fund (CCF-ALIMAMA OF 2025001), Guangdong Basic and Applied Basic Research Foun- dation (2025A1515012933, 2025A1515011732 Shenzhen Science and Technology Program IKQTID202407291101554066 ZDSYS20230626091 203008).
References
[1] Qianli Ma, Zhen Liu, Zhenjing Zheng, Ziyang Huang, Siying Zhu, Zhongzhong Yu, and James T Kwok. A survey on time-series pre-trained models. IEEE Transactions on Knowledge and Data Engineering, 2024.
[2] Luca Benfenati, Daniele Jahier Pagliari, Luca Zanatta, Yhorman Alexander Bedoya Velez, Andrea Acquaviva, Massimo Poncino, Enrico Macii, Luca Benini, and Alessio Burrello. Foun- dation models for structural health monitoring. arXiv preprint arXiv:2404.02944 2024.
[3] Weijie Xia, Gao Peng, Chenguang Wang, Peter Palensky, Eric Pauwels, and Pedro P Vergara. An efficient and explainable transformer- few-shot learning for modeling electricity consumption profiles across thousands of domains. arXiv preprint arXiv:2408. 2024.
[4] Thomas Wolf. Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et Huggingface's s transform- ers: State-of-the-art natural language processing. arXiv preprint urXiv:
2019.
[5] Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, al. Tensorflow: Large-scale machine learning on heterogeneous systems, 2015.
[6] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.
[7] Andrew G Howard. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint 17Xiv: 2017.
[8] Mingxing Tan and Quoc Le. Efficientnet: Rethinking model scaling for convolutional neural networks. International conference on muchine learning. pages 6105-6114 14. PMLR, 2019.
[9] Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer: Generative pre-trained transformers are large time series models. In Forty-first International Conference on Machine Learning. 2024.
[10] Vijay Ekambaram, Arindam Jati, Pankaj Dayama, Sumanta Mukherjee. Nam Nguyen, Wesley M Gifford, Chandra Reddy, and Jayant Kalagnanam. Tiny time mixers (ttms): Fast pre-trained models for enhanced zero/few-shot forecasting of multivariate time series. Advances Neural Information Processing Systems, 37:74147-74181, 2024,
[11] Yuhe Ding, Bo Jiang, Aijing Yu, Aihua Zheng, and Jian Liang. Which model to transfer? a survey transferability estimation. arXiv preprint 15X 2024.
[12] Cuong Nguyen, Tal Hassner, Matthias Seeger, and Cedric Archambeau. Leep: A new measure to evaluate transferability of learned representations. In International Conference on Machine Learning, pages 7294- 7305. PMLR, 2020.
[13] Yandong Li, Xuhui Jia, Ruoxin Sang, Yukun Zhu, Bradley Green, Liqiang Wang, and Boqing Gong. Ranking neural checkpoints. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2663-2673. 2021.
[14] Yajie Bao, Yang Li, Shao-Lun Huang, Lin Zhang, Lizhong Zheng, Amir Zamir, and Leonidas Guibas. information -ewerrrttitthhticcccooo approach to transferability task transfer learning. In 20/9 IEEE international conference image processing (ICIP), pages 2309-2313, IEEE, 2019.
10

---

[15] Huiwen Xu and U Kang. Fast and accurate transferability measurement by evaluating intra-class feature variance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11474-11482 2023.
[16] Wengi Shao, Xun Zhao, Yixiao Ge, Zhaoyang Zhang, Lei Yang, Xiaogang Wang. Ying Shan, and Ping Luo. Not all models are equal: Predicting model transferability in a self-challenging fisher space. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIV, pages 286-302. Springer, 2022.
[17] Kaichao You, Yong Liu, Jianmin Wang, and Mingsheng Long. Logme: Practical assessment of pre-trained models for transfer learning. In International Conference on Machine Learning. pages 12133-122 12143. PMLR, 2021.
[18] Mohsen Gholami, Mohammad Akbari, Xinglu Wang, Behnam Kamranian, and Yong Zhang. Etran: Energy-based based transferability estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 18613-18622, 2023.
[19] Zijian Wang, Yadan Luo, Liang Zheng, Zi Huang, and Mahsa Baktashmotlagh. How far pre- trained models are from neural collapse on the target dataset informs their transferability. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5549-5558, 2023.
[20] Li Dong, Nan Yang, Wenhui Wang, Furu Wei, Xiaodong Liu, Yu Wang, Jianfeng Gao, Ming Zhou, and Hsiao Wuen Hon. Unified language model pre-training for natural language under- standing and generation. Advances in neural information processing systems, 32, 2019.
[21] Javier Selva, Anders S Johansen, Sergio Escalera, Kamal Nasrollahi, Thomas B Moeslund, and Albert Clapés. Video transformers: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, (5(11):
2023.
[22] Ching Chang, Wen-Chih Peng. and Tien-Fu Chen. Llm4ts: Two-stage fine-tuning for time-series forecasting with pre-trained Ilms. arXiv preprint arXiv:2308. 08469, 2023.
[23] Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt-based generative pre-trained transformer for time series forecasting. arXiv preprint 1rxXi::331000444 2023.
[24] Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Hassen, Anderson Schneider, al. Lag-llama: Towards foundation models for time series forecasting. In R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models, 2023.
[25] Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai. Shuo Li, and Artur Dubrawski. Moment: A family of open time-series foundation models. In Forty-first International Confer- on Machine Learning, 2024.
[26] Shanghua Gao, Teddy Koker, Owen Queen, Thomas Hartvigsen, Theodoros Tsiligkaridis, and Marinka Zitikk. Units: A unified multi-task time series model. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[27] Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-moe: Billion-scale time series foundation models with mixture of experts. arXiv preprint arXiv:2409.16044 2024.
[28] Anh T Tran, Cuong V Nguyen, and Tal Hassner. Transferability and hardness of supervised classification tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1405, 2019.
[29] Anthony J. Bagnall, Hoang Anh Dau, Jason Lines, Michael Flynn, James Large, Aaron George Bostrom, Paul Southam, and Eamonn Keogh. The uea multivariate time series classification archive, 2018. arXiv preprint XrXiv: 00075, 2018.

---

[30] Jiexi Liu and Songean Chen. Timesurl: Self-supervised contrastive learning for universal time series representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence. volume 38, pages 13918-13922 2024.
[31] Haixu Wu, Jiehui Xu, Jianmin Wang, and Mingsheng Long. Autoformer: Decomposition transformers with Auto-Correlation for long-term series forecasting. In NeurlPS, 2021.
[32] Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. Noise reduction speech processing, pages 1-4, 2009.
[33] Shiyu Wang, Jiawei Li, Xiaoming Shi, Zhou Ye, Baichuan Mo, Wenze Lin, Shengtong Ju, Zhixuan Chu, and Ming Jin. Timemixer++: A general time series pattern machine for universal predictive analysis. arXiv preprint arXiv:2410. 16032, 2024.
[34] Haixu Wu, Tengge Hu. Yong Liu, Hang Zhou, Jianmin Wang, and Mingsheng Long. Times- net: Temporal 2d-variation modeling for general time series analysis. arXiv preprint arXiv:2210.02186, 2022.
[35] Jaxxian Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. Simmtm: A simple pre-training framework for masked time-series modeling. Advances in Neural Information Processing Systems, 36, 2024.
[36] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton, Similarity of neural network representations revisited. In International conference on machine learning, pages 3519-3529, PMLR, 2019.
[37] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernels based on centered alignment. The Journal of Machine Learning Research, 13:795-828, 2012.
[38] Nello Cristianini, John Shawe-Taylor, Andre Elisseeff, and Jaz Kandola. On kernel-target alignment. Advances in neural information processing systems, 14, 2001.
[39] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Measuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory, pages 63-77. Springer, 2005.
[40] Gene H Golub and Charles F Van Loan. Matrix computations. JHU press, 2013.
[41] Gene H Golub and Henk A Van der Vorst. Eigenvalue computation in the 20th century. Journal of Computational and Applied Mathematics, 123(1-2):35-65, 2000.
[42] Michal Pándy, Andrea Agostinelli, Jasper Uijlings, Vittorio Ferrari, and Thomas Mensink. Trans- ferability estimation using bhattacharyya class separability. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9172-9182. 9182, 2022.
[43] Yuncheng Yang, Meng Wei, Junjun He, Jie Yang. Jin Ye, and Yun Gu. Pick the best pre-trained model: Towards transferability estimation for medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 674-683. Springer, 2023.
[44] Yong Liu, Guo Qin, Xiangdong Huang, Jianmin Wang, and Mingsheng Long. Timer-xl: Long- context transformers for unified time series forecasting. arXiv preprint arXiv:2410.04880 2024.
[45] George B. Moody, Roger G. Mark, and Ary L. Goldberger. Physionet: Physiologic signals, time series and related open source software for basic, clinical, and applied research. EMBC, 2011.
[46] G. Zerveas, Srideepika Jayaraman, Dhaval Patel, Anuradha Bhamidipaty, and Carsten Eickhoff. transformer-based framework for multivariate time series representation learning. KDD, 2021.
12

---

[47] Nan Ding, Xi Chen, Tomer Levinboim, Soravit Changpinyo, and Radu Soricut. Pactran: Pac- bayesian metrics for estimating the transferability of pretrained models classification tasks. In Computer Vision-FCCC 2022: 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part XXXIV, pages 252-268. Springer, 2022.
[48] Mengyu Dai, Amir Hossein Raffice, Aashish Jain, and Joshua Correa. Evaluating transferability in retrieval tasks: An approach using mmd and kemel methods. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2024.
[49] Spyros Makridakis, M4 dataset, 2018. URL httpa///gggtuubb.co/a//aa/Cepapt
M4 Bll--etttod//rre/mmatttt/////aae
[50] Ya Su, Y. Zhao, Chenhao Niu, Rong Liu, W. Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. KDD, 2019.
[51] Kyle Hundman, Valentino Constantinou, Christopher Laporte, Ian Colwell, and Tom Södersstöö. Detecting spacecraft anomalies using istms and nonparametric dynamic thresholding. KDD, 2018.
[52] Aditya P. Mathur and Nils Ole Tippenhauer. Swat: water treatment testbed for research and training on ICS security. In CySWATER, 2016.
[53] Ahmed Abdulaal, Zhuanghua Liu, and Tomer Lancewicki. Practical approach to asynchronous multivariate time series anomaly detection and localization. KDD, 2021.
[54] Hao Li, Pratik Chaudhari, Hao Yang. Michael Lam, Avinash Ravichandran, Rahul Bhotika, and Stefano Soatto. Rethinking the hyperparameters for fine-tuning. arXiv preprint arXiv:2002.11770, 2020.
[55] Maurice G Kendall. A new measure of rank correlation. Biometrika, 30(1/)):1-93, 1938.
[56] Grace S Shich. A weighted kendall's tau statistic. Statistics & probability letters, 177::17224 1998.
13

---

A
Limitations
Our method may perform poorly on some datasets with severe distribution shifts or structural breaks, the results on the training cannot accurately reflect the model's performance the test set. This remains a challenging problem, and no existing method has effectively addressed it yet.
B Broader impacts
The work in this paper contributes facilitating the evaluation of transfer capability for time series pre-trained models, therehy further saving time and training resources and enabling the rapid selection of suitable time series pre-trained models for downstream tasks.
C Experiment Details
C.1 Implement Details
We provide detailed descriptions of the datasets in Tables 8 For all 5 downstream tasks, follow the experimental setup of [34). To compute the transfer performance values, we carefully fine-tumed the pre-trained models through hyperparameter grid search. As (54) highlighted, learning rate and weight decay are the two most critical parameters. Therefore, we performed grid search over learning rates and weight decay values learning rates ranging from 10 3 to 10 and 3 weight decay values from 10-3 to 10-5) to select the optimal hyperparameters. The fine-tuning experiments of the pre-trained models were conducted on an NVIDIA H20 GPU with 96GB of memory. is important to note that we only fine-tuned the pre-trained models to validate the effectiveness of the proposed method; the method itself does not require fine-tuning of the pre-trained models. All the results of pre-trained model transferability evaluation metrics were obtained on an AMD EPYC 7513 32-Core CPU.
C.2 Model Pool Details
We selected 10 pre-trainee pre-trained models from four model families to form the model pool, including MOMENT-small (25). MOMENT-base [25), MOMENT-large [25], Timer-small [9]. Timer-
9.. TimerXL-small [4). TimerXL- 44 UniTS [26]. UniTS base [26]. and UniTS-large
26 As shown in Table the selected pre-trained models differ in structure, model parameters, and the scale of pre-training. The MOMENT models were reconstructed and trained on the large hybrid dataset Time-series Pile; Timer-small and TimerXL-small were pre-trained the ERA5 dataset, while Timer-base and TimerXL-Be --base were pre-trained on the large dataset UTSD; the UniTS series models were pre-trained on multi-domain hybrid dataset for time series. This highlights the differences between the models and further demonstrates that the proposed method is sufficient to support general transferability.
Table 7: The Architecture, Model size, and Scale of the pre-trained data of the model pool.
Method
Timer
Timer-XL
MOMENT
UniTS
Architecture
Decoder
Decoder
Encoder
Encoder
Model size
11M~67M
14M~69M
40M~380M
3M~8.4M
Pre-training Scale
0.5B, 28B
0.5B, 28B
1.13B
0.5B
C.3
Evaluation Metric
To evaluate the effectiveness the proposed method, we refer to previous work [17 18
and use Kendall's 55 to measure the correlation between the estimated evaluation scores and the ground- truth ranking scores. This metric assesses ranking consistency, where larger 7 value indicates greater effectiveness in ranking models. It is defined the number of concordant pairs minus the
14

---

number of discordant pairs, divided by the overall number of pairs (2) as follows:
2
E
sgn(G; Gy).sgn([ T2).
(12)
M(M-1
ISiOSM
Furthermore, we adopt a weighted version of Kendall's r [56]. denoted as Twx to assign higher weights to models ranked higher, to quantify this correlation.
C.4 Power Iteration
Due to the limited space in the main text, we show the detailed procedure of the power iteration method here, shown in Algorithm T
Algorithm 1 Power Iteration
Input: Matrix H, initial vector number of iterations k; here we set 10. Dominant eigenvalue Amax. corresponding eigenvector ti.
Output:
Initialize to randomly
for i =
do
0i+11 = Hu; Normalize U++1 to unit length
end for Estimate the dominant eigenvalue Amux
22 Hoy
D More Experimental Results
D.1 Statistical Significance Testing
To validate the effectiveness of TEMPLATE over ETran, we conducted statistical significance test. The results show that the mean of TEMPLATE (0.483) is significantly higher than that of ETran (0.359), with a mean difference of 124, which is statistically significant statistic 2.679, P P_value 0.012). Therefore, TEMPLATE demonstrates significantly better performance across all tasks compared ETran, confirming its effectiveness improvement.
D.2 Ablation Study
(a) Classification.
(b) Imputation.
(c) Forecasting.
(d) anomaly detection.
Figure 6: Effectiveness each individual component in TEMPLATE. We use the three terms
individually remove them at time from the full system. The results are averaged across all datasets.
Due to space limitations, have included the complete ablation experiment results in Figure The experimental results show that, compared to the three individual metrics, TEMPLATE achieves the best performance, demonstrating the effectiveness of TEMPLATE. Additionally, we have also validated the effectiveness of combining these metrics in pairs. The combination of Sai and Sn, outperforms each metric individually, proving their complementary nature. This demonstrates that evaluating the pre- trainer model's 's ability for time series semantic learning from the perspectives of
15

---

Table 8: Dataset descriptions. The dataset size is organized in (Train, Validation, Test).
Tasks
Dataset
Dim
Series Length
Dataset Size
Information (Frequency)
ETTml, ETTm2
7
(96, 192, 336, 720)
(34465, 11521, 11521)
Electricity (15 mins)
ETTh1, ETTh2
7
(96, 192, 336, 720)
(8545, 2881, 2881)
Electricity mins)
Forecasting
Electricity
321
(96, 192, 336, 720)
(18317, 2633, 5261)
Electricity (Houryy
(Long-term)
Traffic
862
(96, 192, 336, 720)
(12185, 1757, 3509)
Transportation (Hourly)
Weather
21
(96. 192, 336, 7205
(36792, 5271, 10540)
Weather (10 mins)
Yearyyrly
|
(23000, O, 23000)
Demographic
M4-Quarterly
(24000, O, 24000)
Finance
Forecasting
M4-Monthly
18
(48000, 0, 48000)
Industry
(short-term)
M4-Weakly
|
13
(359, 0, 359)
Macro
M4-Daily
14
(4227, 0, 4227)
Miero
M4-Hourly
|
48
(414, 0, 414)
Other
ETTmI,ETTm2
7
96
(34465, 1 11521, 11521)
Electricity (15 mins)
ETTh1, ETTh2
7
(8545, 2881, 2881)
Electricity (15 mins)
Imputation
Electricity
321
(18317, 2633, 5261)
Electricity (15 mins)
Traffic
862
96
(12185, 1757, 3509)
Transportation (Hourly)
Weather
21
96
(36792, 5271, 10540)
Weather (10 mins)
EthaooCConcenttationn
3
1751
(261, O, 263)
Alcohol Industry
FaceDetection
144
|
62
(5890, O, 3524)
Face (250Hz)
Handwriting
3
|
152
(1.50, O, 850)
Handwriting
Classification
AapaneseVowel
12
(270, 0, 370)
Voice
(UEA)
PEMS-SF
963
144
(267, 0, 173)
Transporationn (Daily)
SelfRegulationSCPI
6
|
896
(268, 0, 293)
Health (256Hz)
SellRegulationSC12
7
1152
(200, 0, 180)
Health (256Hz)
SpokenArrabicDigiis
13
|
93
(6599, 0, 2199)
Voice (1 (11025Hz)
UWaveGestureibbray
3
|
315
(120, 0, 320)
Gesture
SMD
38
100
(566724, 141681, 708420)
Server Machine
Anomaly
MSL
.55
100
(44653, 1 11664, 73729)
Spacecraft
Detection
SMAP
25
100
(108146, 27037, 427617)
Spacecraft
SWaT
51
100
(396000, 99000, 449919)
Infrustructure
PSM
25
100
(105984, 26497, 87841)
Server Machine
dependency learning and pattern learning is an effective and reasonable approach. Although Sta did not achieve the best correlation in the anomaly detection task, compared to W/O Sta. TEMPLATE showed an 8% average improvement in to across 5 downstream tasks, effectively proving the validity of St..
Additionally, it is important note that TEMPLATE is very robust method. We conducted additional ablation experiments to explore the specific implementation methods for the scores. For Sa, the main idea is to measure the alignment of the eigenvector corresponding to the largest singular value between the trend component and the original sequence. In the table, we also implemented Sa based on Fuclidean distance, and it can seen that Sai based on Euclidean distance achieves positive transfcrability evaluation. For Sto. the main idea is to measure the similarity between features
16

---

Table 9: Ablation study on the specific implementation methods of Sa and Sto.
Method
EthanolConcentraaion Handwriting PEMS SF FaceDetection
0.332
0.597
0.370
0.361
(a(
0.349
0.608
0.381
0.366
Sta (dot product)
0.421
0.654
0.276
0.450
(HSCH)
0.436
0.698
0.350
0.453
meet the requirements of different task types for features. In the following table, we implemented S, based on squared dot product, and it can be observed that Sta based on squared dot product also higher-dimennional space and then computes the similarity. The method of measuring feature similarity has good properties [36]. Therefore, we use HSIC to measure similarity. The experimental results in the table further demonstrate the effectiveness and robustness of TEMPLATE.
achieves positive transferability evaluation. HSIC maps the features
Table 10: Ablation Experiment on the Sensitivity of Hyperparameter X.
EthenooConnentratton
Handwriting
PEMS SF SelfRegulationSCP2
-0.6
0.716
0.822
0.461
0.551
-0.8
0.724
0.822
0.461
0.551
0.724
0.822
0.470
0.551
Finally, we note that the three scores in the final score are equally important, and À is only related to the task type rather than the specific downstream task dataset. To demonstrate the robustness of our method, add parameter sensitivity experiment for À in Table TO The experimental results show that on specific downstream datasets, there is no need to fine-tune
Table 11:
Short-Term Benchmark:
Performance (Weighted Kendall's Tw) of different methods. The
best results are highlighted in bold.
Datasets
Yearly
Quarterly
Monthly
Others
Average
LogME
0.492
0.332
0.562
0.468
0.464
ETran
0.235
0.763
0.624
0.318
0.485
Ours
0.534
0.667
0.712
0.417
0.583
D.3
Full Result
Due to space limitations the main text, we have included the results of the short-term forecasting, long-term forecasting and imputation in Tables [T] Tables 12 and 13 The experimental results
demonstrate that compared to LogME and ETran, TEMPLATE achieves the best average performance.
Table 12: The full result for forecasting task.
ETTh2
PTTel
BTTx2
woather
RCL.
Toatfic
Lengh
9s 192
316
720
96
193.
33M
720
Ws
192
336
720
197
316
770
197
376
730
96
192
336
770
197
376
720
LogME 0.355 0.0.11200000
0.199
0.9960.230
0.411
233000000
ETrnn 010070276000000 1000 (0.331
0.150 u
0.54
0.208
Ous 3.090.2430005555 w00002200..10coooooma.
0.42
0.20j01880
Table 13: The full result for imputation task.
Tatmens
1TTh
ETT12
BTTml
BTTw2
weather
ECL
Trufff:
Madic Buatn 12.5%. 2914 37 95.
25% 30.55 12.9% 77.5% 13.5% 29% 57.5% 11.5% 2015 52.9% so%
37.95
37,556
Lights:
0.193 641 0.21 1HS 211
W3AS & 0.301
0.342 06913
0.505 0.333 n 3.00p an 195 1.0 a 0.35 in 190 (109) (605)
Einse
0.299
0.384
0.540
W153 0.383 1L39 0451 0451
0L100
0L266
0.669 0.393 0255
0.305
0.44 0439/044 0.42 0.556 0.300 0,590 0.365 0,206 0.50) 0.284 0.490 0.000 0.501
01201
0.536
0,479 0.54 (1985j0.00
01678 0.602)
0.623
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We include detailed information in Section
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how be expected to generalize other settings.
much the results
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations are included in Appendix A
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
18

---

Justification:
We have no relevant theory to prove the need.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We include the detailed experimental settings in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
19

---

Answer: [Yes]
Justification:
The anonymous code link is available at: http://aanoooouss.44m
science/r/TEMPLAN TEE.CC/TEEEEELLLEEEEEEEE AOAA/
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
(osssiiccc)y)) for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https ioweeeomiisiiiioooiooo..o
//nips. //nips.c//pil cc/public/guu
for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We include the detailed experimental settings in appendix C and main text.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We conducted statistical tests to demonstrate the effectiveness of the method.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
20

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We detail the computing resources used in Appendix C.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
In every respect in the paper, we follow the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We include the impact statement in Appendix B
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
21

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All data,model.and code in the paper respect the license.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
22

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not invlove crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
23

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper did not use LLM in method research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
24


Reviews:
[{"review_id": "Kj3uJMwbJH", "reviewer": "Reviewer_4tiJ", "summary": "The paper proposes ​​TEMPLATE​​, a novel framework for evaluating transferability in time-series foundation models. Addressing the lack of tailored metrics for time series (unlike computer vision), TEMPLATE introduces three complementary scores: ​​Dependency Learning Score​​, ​​Pattern Learning Score​​ and Task Adaptation Score. Extensive experiments are conducted across across 32 datasets and 5 downstream tasks, and the result demonstrate the effectiveness of TEMPLATE.", "questions": "See Weaknesses.", "limitations": "Yes.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n1. The paper dedicates transferability metric for time-series foundation models, addressing a critical gap in literature.\n2. The proposed method is benchmarked rigorously across ​​5 distinct tasks​​ and ​​32 datasets​​, including multivariate and large-scale benchmarks (e.g., Traffic: 862 channels).\n3. The proposed method avoids brute-force fine-tuning via power iteration and nuclear norm approximations, reducing computational overhead.\n\n## Weaknesses\n1. Reliance on SVD assumes principal components reliably represent transferable knowledge. This may not hold for noisy or non-stationary data, where lower singular values could be informative.\n2. Dependency Learning Score​​ uses average pooling for trend extraction (Eq. 1), potentially oversimplifying complex temporal dependencies (e.g., multi-scale trends).", "was_revised": true, "final_justification": "The rebuttal has well addressed my concerns.", "rebuttal": "Thank you for your valuable feedback and positive comments. We greatly appreciate your recognition of the effectiveness of the TEMPLATE.\n\n## W1: About the influence of smaller singular values.\n\nWe sincerely appreciate your insightful comment regarding the potential information loss induced by discarding smaller singular values in Singular Value Decomposition (SVD), particularly under non-stationary scenarios. To address this concern, we conducted experiments on non-stationary datasets [1]. During testing, truncated SVD (formula specified below) was applied to the encoder outputs, removing the smallest 20% of singular values (retaining the top 80%), with results compared against the original baseline.\n\nGiven a target rank k (satisfying \\($k \\leq \\text{rank}(A)$\\)), the truncated SVD approximation \\($A_k$\\) is defined as:\n\n$$\nA_k = U_k \\Sigma_k V_k^T\n$$\n\nwhere:\n\n- $U_k \\in \\mathbb{R}^{m \\times k}$: Consists of the first k columns of U (left singular vectors corresponding to the **top k largest singular values**)\n- $\\Sigma_k \\in \\mathbb{R}^{k \\times k}$ :Diagonal submatrix from the first k rows and columns of $\\Sigma$ ($\\text{diag}(\\sigma_1, \\sigma_2, \\ldots, \\sigma_k)$\\).\n- $V_k \\in \\mathbb{R}^{n \\times k}$: Consists of the first k columns of V (right singular vectors corresponding to the **top k largest singular values**)\n\nAs presented in the table below, truncating smaller singular values induces **minimal and consistent accuracy deviations across all datasets and models.** The maximum observed performance degradation is merely **–0.8% (ETTh2 with MOMENT-base)**, while changes in most cases remain below **–0.5%**. Crucially, absolute metric values (e.g., 0.1789 vs. 0.1793 for ETTm2) confirm that predictive performance **remained virtually unaffected**. Empirically, this suggests that smaller singular values contribute negligibly to knowledge transfer within our framework and **can thus be safely omitted.**\n\n\n|       | MOMENT-base            | Timer-base             | UniTS-base             |\n| ----- | ---------------------- | ---------------------- | ---------------------- |\n|       | original｜T SVD｜ratio | original｜T SVD｜ratio | original｜T SVD｜ratio |\n| ETTh1 | 0.3926｜0.3944｜-0.4%  | 0.3772\\|0.3791\\|-0.5%  | 0.3829\\|0.3856\\|-0.7%  |\n| ETTh2 | 0.2837｜0.2862｜-0.8%  | 0.3025\\|0.3033\\|-0.2%  | 0.3022\\|0.3030\\|-0.2%  |\n| ETTm2 | 0.1739｜0.1747｜-0.4%  | 0.1789\\|0.1793\\|-0.2%  | 0.1862\\|0.1871\\|-0.4%  |\n\n## W2: About the impact of average pooling on Dependency Learning Score.\n\nWe sincerely appreciate the reviewer for pointing out the potential risk of oversimplification in average pooling. To evaluate this issue, we conducted a comparative experiment between moving average pooling and **multi-scale decomposition (using wavelet decomposition with db4 basis functions)** for trend extraction tasks.\n\nAs shown in the table, wavelet decomposition **significantly reduces prediction accuracy** (e.g., causing a **15.3% performance drop** on the EthanolConcentration dataset). This indicates that **coarse-grained information extracted through multi-scale analysis fails to enhance prediction effectiveness.** Simultaneously, average pooling not only avoids complex transformation processes but also exhibits higher computational efficiency. Therefore, the method adopted in Equation (1) **achieves an optimal balance between accuracy and efficiency for the current task.**\n\n\n| Dataset              | **moving averages decomposition**（$τ_w$） | **wavelet decomposition**（$τ_w$） |\n| -------------------- | ------------------------------------------- | ----------------------------------- |\n| EthanolConcentration | 0.346                                       | 0.293                               |\n| Handwriting          | 0.600                                       | 0.466                               |\n| UWaveGestureLibrary  | 0.189                                       | 0.105                               |\n\n[1] Non-stationary transformers: Exploring the stationarity in time series forecasting. NeurIPS, 2022."}, {"review_id": "ypDiTfRNzF", "reviewer": "Reviewer_shCY", "summary": "This paper introduces TEMPLATE, a novel framework for estimating the transferability of time series pre-trained models. TEMPLATE consists of three complementary metrics—Dependency Learning Score, Pattern Learning Score, and Task Adaptation Score—designed to address unique challenges in time series, such as cross-task heterogeneity and feature space mismatch. The framework demonstrates strong empirical performance on five mainstream time series tasks, outperforming prior transferability estimation methods such as ETran and LogME, while maintaining scalability via efficient approximations like power iteration.", "questions": "1. Have you considered learning the weights of the three metrics instead of using a fixed aggregation with task-specific sign? Would a learned or adaptive approach improve performance or robustness?\n2. Beyond Kendall's $\\tau_w$, how well does TEMPLATE perform in practical top-k model selection scenarios (e.g., top-1 accuracy or top-3 hit rate)?\n3. Performance on datasets like PSM and SWaT appears to lag behind ETran. Could you provide insights into what type of distribution shift harms TEMPLATE, and how it might be mitigated?\n4. Do the three sub-scores interact or overlap in what they measure? For instance, does high dependency learning score already imply good pattern learning?\n5. Can TEMPLATE be extended to or applied in zero-shot or few-shot learning scenarios, which are increasingly relevant in time series foundation model usage?", "limitations": "Yes.  The paper discusses the limitations in handling datasets with severe distribution shifts or structural breaks, and acknowledges that this remains a challenging open problem.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths:**\n\n1.  The methodology is sound, with clear mathematical formulations and robust benchmarking across a wide range of datasets. The use of power iteration for scalability is a practical engineering contribution.\n2.  The proposed metrics are designed specifically for time series, filling a clear gap in existing literature focused primarily on vision tasks.\n3.  The paper is generally well-written with logical structure, clear figures, and detailed appendices.\n\n**Weaknesses:**\n\n1.  While the three proposed scores (Dependency Learning Score, Pattern Learning Score, and Task Adaptation Score) are intuitively motivated and empirically validated, the paper lacks a theoretical analysis or formal justification to support why these metrics correlate well with transferability across diverse tasks.\n2.  The final transferability score combines three normalized metrics with fixed equal weights (and a task-type-based sign for $\\lambda$). However, the choice of this fixed linear aggregation lacks theoretical grounding or empirical sensitivity analysis. There may be cases where certain metrics dominate or interfere depending on task characteristics.\n3.  The main quantitative evaluation focuses on Kendall's $\\tau_w$, which reflects ranking correlation but does not capture real-world model selection performance such as top-1 accuracy or top-k hit rate. Users in practice may care more about how often the top-ranked model is truly best-performing.\n4.  The performance of TEMPLATE lags behind ETran on certain anomaly detection datasets (e.g., PSM and SWaT in Table 6), suggesting a degree of sensitivity to dataset-specific characteristics or distribution shifts.  Since real-world time series anomaly detection often involves non-stationary data or rare events, further analysis of failure modes or robustness under distribution shift would strengthen the method's practical applicability.", "was_revised": true, "final_justification": "After considering the rebuttal and discussion:\n\nResolved:\n\nThe authors provided empirical evidence and literature references to support the three metrics’ link to transferability.\n\nCorrelation and sensitivity analyses justify the fixed-weight aggregation design.\n\nAdditional Top-k selection results and few-shot experiments strengthen practical relevance.\n\nAnomaly detection performance was explained with clear dataset-specific factors.\n\nUnresolved:\n\nThe method still lacks a formal theoretical framework linking the metrics to transfer performance beyond empirical justification.\n\nRobustness under significant distribution shifts remains insufficiently explored.\n\nGiven the novelty, clear contribution to time series transferability estimation, and the improved empirical support provided in the rebuttal, I maintain a borderline accept recommendation, with the remaining concerns being directions for future research rather than fatal flaws.", "rebuttal": "Thank you for your valuable feedback and positive comments. We greatly appreciate your recognition of the  writing and methodology.\n\n## W1: About the connection between the three scores and transferability.\n\nRegarding the **Pattern Learning Score**, [1] demonstrates that larger singular values encapsulate more transferable knowledge, enabling superior transferability—a well-established principle in transfer learning. For the **Dependence Learning Score**, this metric is grounded in fundamental time-series principles [2] [3], where **long-term dependencies reside in trend components, reflecting core model design philosophies [4].** Our SVD analysis reveals that models effectively capturing dependencies exhibit high alignment between principal components and trend features. Since transfer capability relies on preserving critical temporal relationships, this alignment serves as a robust indicator of transferability. Concerning the **task adaptation score**, as documented in [5] [6], distinct tasks require specialized representations (hierarchical features for classification/imputation tasks versus low-level features for forecasting/anomaly detection). **This score evaluates a model’s capacity to generate task-adapted representations.** We will strengthen these theoretical foundations in the final manuscript.\n\n## W2&Q1&Q4: About the basis and sensitivity analysis of fixed equal-weight linear aggregation, as well as adaptive learnable weights.\n\nRegarding the linear aggregation of the three scores, we incorporated a correlation analysis among them. Specifically, Pearson and Spearman correlation coefficients were computed across all classification task datasets for the three metrics (as tabulated below). **The results indicate consistently low correlation magnitudes**, with absolute values of both Pearson and Spearman coefficients below 0.3. This demonstrates the **absence of strong inter-score correlations and confirms their relative orthogonality.** Consequently, **linear weighted aggregation of the three scores is statistically justified.**\n\n\n|                     | Pearson correlation | Spearman correlation |\n| ------------------- | ------------------- | -------------------- |\n| $S_{dl}$ - $S_{pl}$ |  0.05                |  0.21                 |\n| $S_{dl}$ - $S_{ta}$ | -0.23               | -0.20                |\n| $S_{ta}$ - $S_{pl}$ | -0.15               | -0.18                |\n\nRegarding the fixed-weight design, we contend that the three scoring metrics are **equally important**. Furthermore, TEMPLATE exhibits strong robustness as a method, eliminating the need for specialized fine-tuning of metric weights. In Table 10 of the Appendix, we include a sensitivity analysis of weighting combinations, demonstrating that variations in the composite weights of the three scores have **minimal impact** on the final TEMPLATE score. Additionally, we implemented adaptive weight allocation using the entropy weight method [7] (as tabulated below). Results indicate that compared to the fixed-weight design with sign functions, **the adaptive weighting strategy does not yield superior performance.**\n\n\n| Dataset                | EthanolConcentration（$τ_w$） | Handwriting（$τ_w$） | UWaveGestureLibrary（$τ_w$） |\n| ---------------------- | ------------------------------ | --------------------- | ----------------------------- |\n| **entropy weight [7]** | 0.696                          | 0.799                 | **0.719**                     |\n| **fixed weights**      | **0.724**                      | **0.822**             | **0.719**                     |\n\n## W3&Q2: About Top1 accuracy and Topk hit rate.\n\nWe supplement the model's **Top-1 accuracy** and **Top-3 hit rate** on classification datasets (as tabulated below). Empirical results demonstrate that TEMPLATE achieves **consistently superior performance** in both Top-1 and Top-3 hit rates compared to alternative methods, further validating its effectiveness.\n\n\n| Method         | TMI | ETran | PACTran | TEMPLATE |\n| :------------- | --- | ----- | ------- | -------- |\n| TOP 1 accuracy | 44% | 11%   | 44%     | 55%      |\n| TOP 3 hit rate | 55% | 21%   | 74%     | 78%      |\n\n## W4&Q3: About the performance of TEMPLATE on some anomaly detection datasets.\n\n**Critically, our method (0.236) demonstrates superior overall performance to ETran (0.187) in anomaly detection tasks, achieving a 26% relative improvement**—substantially surpassing the 6% gain over prior SOTA reported in ETran's original paper [8] for regression tasks. The table below details relative improvements and SOTA attainment rates across methods, revealing TEMPLATE's **dominance in both metrics**. Notably, anomaly detection exhibits the **lowest evaluation accuracy** among all transfer tasks (imputation/forecasting/classification), underscoring inherent challenges within this domain.\n\nFurthermore, we observe that the underperforming SWaT and SMD datasets **contain no annotated anomalous samples in their training sets**. This implies that during transfer evaluation, models are **completely unexposed to anomalous patterns**; consequently, the derived feature matrices are **constructed solely from normal data**, devoid of any representation of anomalies. Conversely, the better-performing MSL and SMAP datasets include partial anomalous samples in their training sets, which **affords models opportunities to perceive anomalous patterns**. We attribute the primary challenge for TEMPLATE on SWaT/SMD to **this fundamental disparity in training data composition**: when training data entirely lacks anomalous exemplars, the model's capacity to detect rare/unseen anomalies is **inherently constrained**. Crucially, ETran primarily focuses on quantifying whether datasets are in-distribution (ID) or out-of-distribution (OOD) relative to pre-trained models, **without granular analysis of feature matrices**, rendering it less sensitive to such specific data deficiencies.\n\nAnomaly detection inherently depends on the ability to identify unseen, rare events. The pervasive absence of such events within the data used for transfer evaluation presents a persistent and unique challenge for this task.\n\n\n| Method                         | GBC [9] | CC-FV [10] | NCTI [11] | ETran [8] | TEMPLATE |\n| ------------------------------ | ------- | ---------- | --------- | --------- | -------- |\n| SOTA Rate in Original Paper    | 37%     | 36%        | 40%       | 45%       | **46%**  |\n| Improvement (vs Previous SOTA) | 20%     | 9%         | 21%       | 18%       | **35%**  |\n\n## Q5: About the application capability of TEMPLATE in zero-shot and few-shot scenarios.\n\nIn zero-shot scenarios, where models lack access to training data, evaluating the transfer capability of pre-trained models **holds limited significance** when directly tested on the target dataset without fine-tuning. For few-shot scenarios, systematic evaluations were conducted on classification datasets. As tabulated below, TEMPLATE is compared with alternative methods under both zero-shot and few-shot settings. Results demonstrate that TEMPLATE achieves **statistically superior accuracy** in both scenarios compared to baseline approaches, confirming its applicability for few-shot learning scenarios.\n\n\n| Dataset              | LogME（$τ_w$）     | ETran（$τ_w$）     | TEMLATE（$τ_w$）   |\n| -------------------- | ------------------- | ------------------- | ------------------- |\n|                      | Zero-shot\\|Few-shot | Zero-shot\\|Few-shot | Zero-shot\\|Few-shot |\n| EthanolConcentration | -0.37\\|0.41         | -0.22\\|0.53         | -0.08\\|0.61         |\n| Handwriting          | 0.07\\|-0.21         | -0.12\\|0.19         | 0.15\\|0.69          |\n| UWaveGestureLibrary  | -0.16\\|0.33         | 0.02\\|0.27          | 0.08\\|0.54          |\n\n[1] Transferability vs. discriminability: Batch spectral penalization for adversarial domain adaptation. ICML, 2019.\n\n[2] STD: a seasonal-trend-dispersion decomposition of time series. TKDE, 2023.\n\n[3] Multilevel wavelet decomposition network for interpretable time series analysis. KDD, 2018.\n\n[4] Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. NeurIPS, 2021.\n\n[5] Timesnet: Temporal 2d-variation modeling for general time series analysis. ICLR, 2023.\n\n[6] Timemixer++: A general time series pattern machine for universal predictive analysis. ICLR, 2025.\n\n[7] Effectiveness of entropy weight method in decision‐making. Mathematical problems in Engineering, 2020.\n\n[8] Etran: Energy-based transferability estimation. ICCV, 2023.\n\n[9] Transferability estimation using bhattacharyya class separability. CVPR, 2022.\n\n[10]  Towards transferability estimation for medical image segmentation. MICCAI, 2023\n\n[11] How far pre-trained models are from neural collapse on the target dataset informs their transferability. ICCV, 2023."}, {"review_id": "07DfASTuIj", "reviewer": "Reviewer_p5ui", "summary": "In this paper, the authors propose three specialized metrics for evaluating the transferability of time-series pre-trained models, specifically designed for time series foundation models. These include: (1) the Dependency Learning Score, which quantifies a model’s ability to capture temporal dependencies; (2) the Pattern Learning Score, which assesses the representation quality in extracting discriminative temporal patterns; and (3) the Task Adaptation Score, which measures cross-task generalization capability. These evaluation metrics are applicable to both classification and regression cases. The authors validate the effectiveness of these metrics on five downstream tasks and demonstrate their superior performance compared to the state-of-the-art ETran method.", "questions": "- The authors claim that \"The largest singular value typically corresponds to the primary temporal pattern in time series\" (line 180). Could you clarify the formal definition of temporal pattern in this context? What are its potential manifestations (e.g., trends, periodicity, or other temporal structures)?\n- There are issues with Equation (5). First, in the conventional sense, $\\mathrm{tr}(A)$ is not equivalent to the Frobenius norm of $A$. Additionally, here $H^1$ represents the output of the first neural network layer, while $H$ denotes the final layer's output. If their dimensions differ, then the computation of $H^1*H^T$ would be infeasible. Are there additional computational steps not explicitly stated here?\n- Regarding Equation (10), how are $\\min(S_{dl})$ and $\\max(S_{dl})$ computed? Based on the earlier description, $S_{dl}$ appears to be calculated from a batch of data. Does this imply that the min and max values correspond to the minima and maxima across multiple batches?\n- The UEA archive contains nearly 30 datasets, but only 9 are selected for the experimental section (line 274). What are the selection criteria? How did the metrics perform on the other datasets?", "limitations": "The authors adopt Kendall's $\\tau$ as the evaluation metric for the proposed measurement method in this paper. The experimental results show relatively low $\\tau$ values, which implies that—with high probability—the truly optimal model may not be the one ranked highest by $\\tau$. From a practical standpoint, the core question we typically care about is: Which model is the best among a pool of candidates? Consequently, the accuracy rate of identifying the top-performing model could serve as a more meaningful evaluation criterion than $\\tau$ itself.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n- The authors address a problem of significant importance by developing specialized model transferability evaluation methods tailored for time series characteristics. \n- The proposed approach features computationally simple and efficient logic.\n- The paper is clearly written and accessible.\n\n### Weaknesses\n- Some formulas in the paper contain flaws, as referenced in the  **Questions** section below.\n- The metrics adopted in the experimental section are not entirely reasonable, as discussed in the **Limitations** section below.", "was_revised": true, "final_justification": "The authors have responded to and addressed all the questions I raised. For Q1, the second and third experiments are convincing. For Q2, the authors clearly explained the details of each equation. For Q3, the author explained the reasons related to time constraints and promised to supplement and refine the experiments in subsequent work.", "rebuttal": "Thank you for your valuable feedback and positive comments. We greatly appreciate your recognition of the importance of the issue and your view that the TEMPLATE is highly efficient.\n\n## Q1: About Definition of Temporal Patterns.\n\nIn this context, we define temporal patterns as follows [1]: **Temporal Pattern refers to a regularity pattern in time series that spans multiple time steps, characterized by statistical features (such as periodicity, trends, etc.) manifested through data points arranged in a specific sequence.** It emphasizes the temporal invariance of the pattern along the time axis.\n\n## Q2: About Statement on Formula Calculation.\n\n1、Thank you for your careful review. We have revised Equation (5) – indeed, the original formulation omitted vectorization operations which invalidated the equality. We will correct the errors in both Equation (5) and Equation (6) as shown below.\n\n$$\n\\langle \\text{vec}(H^1 (H^1)^T), \\text{vec}(H H^T) \\rangle = \\text{tr}(H^1 (H^1)^T H H^T) = \\|H^T H^1\\|_F^2 \\quad (5)\n$$\n\n$$\n\\frac{1}{(n - 1)^2} \\text{tr}(H^1 (H^1)^T H H^T) = \\|\\text{cov}((H^1)^T, H^T)\\|_F^2  \\quad (6)\n$$\n\n2、Regarding $\\mathbf{H}^1 * \\mathbf{H}^T$: Since our focus is on the encoder structure – which consists of **$n$ stacked identical layers** – $\\mathbf{H}^{(1)}$ denotes the output of the first layer while $\\mathbf{H}^{(L)}$ represents the final-layer output. **This architectural consistency ensures dimensional uniformity across all layers.**\n\n## Q3: About the calculation logic of the $S_{dl}$ score.\n\nWe clarify the computational logic of  $S_{dl}$: this score represents the **mean value across all batches for a single model on a specific dataset**. In Equation (10), $min( S_{dl})$ and $max( S_{dl})$ denote the **minimum and maximum values of $S_{dl}$ scores across all models evaluated on the identical dataset**. By performing this cross-model normalization, we scale the three scores to **comparable dimensions** for unified computation.\n\n## Q4: About the dataset selection criteria for classification tasks and the performance on other classification datasets.\n\nWe rigorously adhere to the experimental setup of Time Series Library (TSLib) [2]—a widely recognized benchmark repository in time series analysis—for evaluations across **five downstream tasks**. Given that TSLib employs standardized datasets, we maintain its data selection protocol to ensure methodological consistency. Additionally, supplementary performance metrics are provided on extended datasets from the UEA benchmark repository (as tabulated below). Results demonstrate that TEMPLATE **achieves statistically significant superiority** over both Etran and LogME across UEA datasets, further substantiating its efficacy.\n\n\n| Dataset            | LogME（$τ_w$） | ETran（$τ_w$） | TEMLATE（$τ_w$） |\n| ------------------ | --------------- | --------------- | ----------------- |\n| AtrialFibrillation | 0.09            | 0.25            | 0.46              |\n| BasicMotions       | 0.23            | 0.51            | 0.69              |\n| Libras             | -0.02           | 0.19            | 0.31              |\n\n## Q5: About Top-1 accuracy and Top-3 hit rate.\n\nWe supplement the model's **Top-1 accuracy** and **Top-3 hit rate** on classification datasets (as tabulated below). Empirical results demonstrate that TEMPLATE achieves **consistently superior performance** in both Top-1 and Top-3 hit rates compared to alternative methods, further validating its effectiveness.\n\n\n| Method         | TMI | ETran | PACTran | TEMPLATE |\n| :------------- | --- | ----- | ------- | -------- |\n| TOP-1 accuracy | 44% | 11%   | 44%     | **55%**  |\n| TOP-3 hit rate | 55% | 21%   | 74%     | **78%**  |\n\n[1] Temporal pattern attention for multivariate time series forecasting. Machine Learning, 2019.\n\n[2] Deep time series models: A comprehensive survey and benchmark. Arxiv, 2024."}, {"review_id": "G9HAgyH6IV", "reviewer": "Reviewer_2eyR", "summary": "This paper addresses the underexplored but important problem of evaluating the transferability of pre-trained models in time series tasks. The authors propose TEMPLATE, a framework that combines three complementary metrics: Dependency Learning Score, Pattern Learning Score, and Task Adaptation Score, to assess the capability of a model to transfer across tasks. The method is validated on 32 datasets across five types of downstream tasks (classification, short- and long-term forecasting, imputation, and anomaly detection), showing consistent performance gains over existing methods such as ETran and LogME.", "questions": "1. Have you considered learning the task-specific λ value adaptively rather than setting it manually based on task type?\n2. How does TEMPLATE perform on non-stationary time series with structural breaks or regime shifts? Would a dynamic trend extractor help?\n3. It would be informative to include model architecture ablations (e.g., Transformer vs CNN) to assess architecture-agnostic robustness.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Problem significance: Transferability estimation for time series pre-trained models is a crucial but overlooked problem, which this paper tackles directly.\n2. Methodological novelty: The paper proposes three task-agnostic and interpretable metrics based on temporal dependency, dominant pattern learning, and hierarchical adaptation.\n3. Extensive experiments: The study covers 5 tasks, 32 datasets, and 10 pre-trained models, with clear benchmarking against 8 prior methods.\n4. Thorough analysis: Ablation studies show complementary benefits of all metrics; time complexity analysis highlights efficiency and scalability.\n\nWeaknesses:\n1. The framework evaluates only static representations, without considering the interaction with fine-tuning dynamics (e.g., sensitivity to learning rate).\n2. While the trend decomposition is intuitive, it may not perform well on non-stationary series with structural breaks; this is not discussed in detail.\n3. The use of a manually-set λ coefficient for task-specific weighting (±1) may limit generality; adaptive or learned weighting would be more flexible.\n4. Anomaly detection performance on some datasets (e.g., SMD) is suboptimal, but the reasons are not fully explored in the main text.", "was_revised": true, "final_justification": "The author's response based on data has addressed my concerns very well. I would like to increase my score to 4 to support the acceptance of this paper.", "rebuttal": "Thank you for your valuable feedback and positive comments. We greatly appreciate your recognition of the importance of the issue and your view that the TEMPLATE method is innovative.\n\n## W1: About the consideration of the dynamic characteristics of fine-tuning.\n\nIn the field of evaluating the transferability of pre-trained models, existing studies[1] [2] [3] generally focus solely on the static performance of models. This is because the interaction of dynamic characteristics during model fine-tuning determines the upper limit of model performance, which serves as the ground truth. The core objective of this research field is to approximate this ceiling, thereby enabling optimal model selection **without dynamic fine-tuning**. Consequently, **our study  confines its scope to assessing static representations, excluding analysis of dynamic characteristic interactions during fine-tuning.**\n\n## W2&Q2: About the performance on non-stationary datasets and the use of the Dynamic Trend Extractor.\n\nWe evaluated the proposed method on **domain-prevalent** non-stationary datasets [4], employing the Augmented Dickey-Fuller (ADF) statistic for **precise quantification** of non-stationarity intensity. Empirical analysis confirms TEMPLATE exhibits **statistically significant performance gains** over ETran under **extreme non-stationarity**. Non-stationary sequences with structural breaks are currently regarded as **highly challenging scenarios**. In the limitations section, we explicitly acknowledge that datasets exhibiting severe distribution shifts or structural breaks remain an **open research problem**.\n\n\n|       | ADF   | ETran（$τ_w$） | TEMPLATE($τ_w$) |\n| :---- | :---- | :-------------- | ---------------- |\n| ETTh2 | -4.13 | 0.197           | **0.240**        |\n| ETTm2 | -5.66 | 0.189           | **0.518**        |\n\nFurthermore, we evaluated whether employing a **dynamic trend extractor** (utilizing db4 wavelet decomposition) could enhance the assessment of Dependence Learning Score. As tabulated below, dynamic trend extraction **fails to yield enhanced outcomes**. Simultaneously, moving averages not only avoids complex transformation processes but also exhibits higher computational efficiency. **Therefore, the method adopted moving averages achieves an optimal balance between accuracy and efficiency for the current task.**\n\n\n| Dataset              | **moving averages decomposition（$τ_w$）** | **wavelet decomposition**（$τ_w$） |\n| -------------------- | ------------------------------------------- | ----------------------------------- |\n| EthanolConcentration | 0.346                                       | 0.293                               |\n| Handwriting          | 0.600                                       | 0.466                               |\n| UWaveGestureLibrary  | 0.189                                       | 0.105                               |\n\n## W3&Q1: About adaptive learning weights.\n\nWe attempted to use the entropy weight method [5] for adaptive weight allocation (as shown in the table below). **The results indicate that compared with the design of fixed weights plus sign function, the adaptive weighting strategy did not achieve better performance.**\n\n\n| Dataset                | EthanolConcentration（$τ_w$） | Handwriting（$τ_w$） | UWaveGestureLibrary（$τ_w$） |\n| ---------------------- | ------------------------------ | --------------------- | ----------------------------- |\n| **entropy weight [5]** | 0.696                          | 0.799                 | 0.719                         |\n| **fixed weights**      | 0.724                          | 0.822                 | 0.719                         |\n\n## W4: About the performance of TEMPLATE on some anomaly detection datasets.\n\n**Critically, our method (0.236) demonstrates superior overall performance to ETran (0.187) in anomaly detection tasks, achieving a 26% relative improvement**—substantially surpassing the 6% gain over prior SOTA reported in ETran's original paper [2] for regression tasks. The table below details relative improvements and SOTA attainment rates across methods, revealing TEMPLATE's **dominance in both metrics**. Notably, anomaly detection exhibits the **lowest evaluation accuracy** among all transfer tasks (imputation/forecasting/classification), underscoring inherent challenges within this domain.\n\nFurthermore, we observe that the underperforming SWaT and SMD datasets **contain no annotated anomalous samples in their training sets**. This implies that during transfer evaluation, models are **completely unexposed to anomalous patterns**; consequently, the derived feature matrices are **constructed solely from normal data**, devoid of any representation of anomalies. Conversely, the better-performing MSL and SMAP datasets include partial anomalous samples in their training sets, which **affords models opportunities to perceive anomalous patterns**. We attribute the primary challenge for TEMPLATE on SWaT/SMD to **this fundamental disparity in training data composition**: when training data entirely lacks anomalous exemplars, the model's capacity to detect rare/unseen anomalies is **inherently constrained**. Crucially, ETran primarily focuses on quantifying whether datasets are in-distribution (ID) or out-of-distribution (OOD) relative to pre-trained models, **without granular analysis of feature matrices**, rendering it less sensitive to such specific data deficiencies.\n\nAnomaly detection inherently depends on the ability to identify unseen, rare events. The pervasive absence of such events within the data used for transfer evaluation presents a persistent and unique challenge for this task.\n\n\n| Method                         | GBC [6] | CC-FV [7] | NCTI [2] | ETran [3] | TEMPLATE |\n| ------------------------------ | ------- | --------- | -------- | --------- | -------- |\n| SOTA Rate in Original Paper    | 37%     | 36%       | 40%      | 45%       | **46%**  |\n| Improvement (vs Previous SOTA) | 20%     | 9%        | 21%      | 18%       | **35%**  |\n\n## Q3: About the ablation Study on the added model structure.\n\nCurrent time series pre-training models universally employ a **Transformer-based architecture** implemented via stacked Transformer layers. To our knowledge, there are currently no large-scale pre-trained models based on CNN. We pre-training of TimesNet (a CNN architecture) on ERA5-Large dataset and integrated it into the model zoo to **evaluate architecture-agnostic robustness**. As tabulated below, TEMPLATE consistently retains **leading accuracy, further demonstrates the superiority of TEMPLATE.**\n\n\n| Dataset              | LogME（$τ_w$） | ETran（$τ_w$） | TEMPLATE（$τ_w$） |\n| -------------------- | --------------- | --------------- | ------------------ |\n| EthanolConcentration | 0.567           | 0.616           | **0.724**          |\n| Handwriting          | -0.445          | 0.427           | **0.792**          |\n| UWaveGestureLibrary  | 0.411           | 0.624           | **0.679**          |\n\n## Q4: About paper formatting concerns.\nThank you for your careful review of our paper's formatting. Regarding the two-column layout issue you noted on page 2, we wish to clarify that **both columns consist entirely of figures (containing no body text)**.\n\n[1] Logme: Practical assessment of pre-trained models for transfer learning. ICML, 2019.\n\n[2] How far pre-trained models are from neural collapse on the target dataset informs their transferability. ICCV, 2023.\n\n[3] Etran: Energy-based transferability estimation. ICCV, 2023.\n\n[4] Non-stationary transformers: Exploring the stationarity in time series forecasting. NeurIPS, 2022.\n\n[5] Effectiveness of entropy weight method in decision‐making. Mathematical problems in Engineering, 2020.\n\n[6] Transferability estimation using bhattacharyya class separability. CVPR, 2022.\n\n[7]  Towards transferability estimation for medical image segmentation. MICCAI, 2023"}]

Author rebuttal:

Meta-review: This well-written paper presents a novel way of assessing transferability in time-series foundation models. It has been evaluated by 4 knowledgeable reviewers who all agreed in their final evaluations that it is acceptable for NeurIPS (2 straight accept scores, 2 marginal accepts), despite noted weaknesses. Most of the weaknesses and doubts raised by the reviewers in the initial round had been sufficiently addressed in the rebuttal. The remaining concerns that have not been addressed include lacking theoretical support and assessment of robustness of the method to distribution shifts. However, the reviewers agree that these issues could be left for future work. The authors are strongly encouraged to carefully reflect on the feedback received in the final revision of the paper.
Decision: Accept (poster)