Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
Authors: Yihang Wang
Yuying Qiu
Peng Chen
Kai Zhao
Yang Shu
Zhongwen Rao
Lujia Pan
Bin Yang
Chenjuan Guo
Venue: ICML 2025 poster (2025)
Keywords: Foundation models; Time series forecasting
Primary area: deep_learning->sequential_models_time_series

Abstract: With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained on multi-source datasets to support diverse downstream prediction scenarios. Existing time series foundation models primarily focus on scaling up pre-training datasets and model sizes to enhance generalization performance.  In this paper, we take a different approach by addressing two critical aspects of general forecasting models: (1) how to derive unified representations from heterogeneous multi-domain time series data, and (2) how to effectively capture domain-specific features to enable adaptive transfer across various downstream scenarios. To address the first aspect, we propose Decomposed Frequency Learning as the pre-training task, which leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series, resulting in unified representations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. Our model achieves the state-of-the-art forecasting performance on seven real-world benchmarks, demonstrating remarkable few-shot and zero-shot capabilities.

Towards a General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
Yihang Wang" I
Yuying Qiu 1
Peng Chen
Kai Zhao
Yang Shu Zhongwen Rao
Lujia Pan Bin Yang'
Chenjuan Guo
Abstract
economics (Qiu et al., 2024; Wu et al., 2025a; Qiu et al., 2025b). However, training deep learning models for each specific dataset is ressuucce-nttnssig and requires tailored parameter tuning. This approach often suffers from limited prediction accuracy due data scarcity (Liu et al., 2024; Wu al., 2021b; 2024; 2025c). A promising solution is to pre-train general model on diverse time series datasets, which can then be fine-tuned with minimal data for different downstream scenarios or even used directly without fine- tuning. Following this idea, foundation models for time series forecasting have gained significant attention. Recent efforts have focused on scaling up pre-training datasets and model sizes enhance generalization performance (Woo et al., 2024; Goswami et al., 2024; Ansari al., 2024). How- ever, excessive scaling introduces high computational costs during training and inference, undermining the practical- ity of general models, particularly in resource-constraingg settings. Beyond scaling, the design of general time series forecasting models can also be approached through pre- training tasks and downstream transfer adaptation. From these two perspectives, identify the following challenges.
With the growing availability of multi-domain time series data, there is an increasing demand for general forecasting models pre-trained multi- source datasets to support diverse downstream pre- diction scenarios. Existing time series foundation models primarily focus on scaling up pre-trate pre-training datasets and model sizes to enhance generaliza- tion performance. In this paper, we take differ- ent approach by addressing two critical aspects of general forecasting models: (1) how derive unified representations from heterogeneous multi- domain time series data, and (2) how to effectively capture domain-specific features to enable adap- tive transfer across various downstream scenarios. To address the first aspect, we propose Decom- posed Frequency Learning as the pre-training task, which leverages trequency-basee masking and re- construction to decompose coupled semantic in- formation in time series, resulting in unified repre- sentations across domains. For the second aspect, we introduce the Time Series Register, which captures domain-specift representations during pre-training and enhances adaptive transferabil- ity downstream tasks. Our model achieves the state-of-the-ard forecasting performance on seven real-world benchmarks, demonstrating remark- able few-shot and zero-shot capabilities.
Obtaining a unified representation from time series data across various domains is challenging. Time series from each domain involve complex temporal patterns, composed of multiple frequency components combined with cach other (Zhou et al., 2022; Wu al., 2025b), which is fre- quency superposition. As shown in Figure (()), different fre- quency components contain distinct semantic information. For example, low and high-frequency components represent long-term trends and rapid variations, respectively (Zhang et al., 2022). Furthermore, different datasets exhibit di- verse frequency distributions, and the significance of low- frequency and nigh-frequency components for time series modeling varies across domains(Zhang al., 2024). As result, large-scale time series data from different domains introduce even more complex temporal patterns and fre- quency diversity. Existing pre-training frameworks (Dong et al., 2024; Nie et al., 2022; Lee et al., 2023), such as masked modeling and contrastive learning, were proposed to learn unified representation from time domain. How- ever, these methods overlook the frequency diversity and complexity exhibited in heterogeneous time series that come
1. Introduction
Time series forecasting plays a crucial role in various do- mains, including energy, smart transportation, weather, and
'Equal contribution 'East China Normal University, Shang- hai, China 'Aalborg University, Aalborg, Denmark 'Huawei Noah's Ark Lab, Shenzhen, China. Correspondence to: Zhongwen Rao <raozhongwen @ thuawei.com>, Chenjuan Guo <cjguo@ www.se.ccm.cuu.cc.....
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Direct Trausfer
Adaptive Transfer
Geranaliodd Repreeantation
Frequency
Finance
Torget Domain
Dirtctt Transfar
U
Traffic
Energy
Spezilic
Represcrnaa
AA original Series
Domain
Adaptive
Healt
Transfer
Frequency superposition Multi-domain
(Tansport)
(Nature)
Electriiity (Lnesgy)
(a) the illustration of general time series forecasting
(b) t-SNE visualization
Figure 1. (a) Pre-training on multi-domain datasets that exhibit combined frequency. Existing general time series forecasting models only extract generalized representations for direct transfer to various downstream larget domains. We propose to learn generalized and specific representations during pre-training, and adaptively transfer them to each target domain. (b) The t-SNE visualization of the hidden representations after direct transfer and adaptive transfer: In direct transfer, representations of different domains are mixed, but in adaptive transfer, they show a clear clustering pattern. The detailed experiment setting in the Appendix A.1.3
from various domains, making it difficult to capture intricate patterns, thus limiting their generalization capabilities.
eralized representations. Second, we introduce Time Series Register (TS- (eSgRegist to learn domain-specifis information in multi-domain data. By setting up register, we generate register tokens to learn each domain-specifie information during pre-training. In a downstream scenario, the model adaptively selects Top-K vectors from the register that are close to the target domain of interest. During fine-tuning, we adjust the selected register tokens with novel learn- able low-rank matrix, which complements target-specific information to perform more flexible adaptive transfer. As shown in Figure 1(b), adaptive transfer successfully utilizes domain-specific information in multi-domain time series, which contributes to the model's performance in target tasks. The contributions are summarized as follows:
Adaptive transferring information from multi-domain time series to specific downstream scenarios presents a challenge. Multi-source time series data originate from various domains (Woo et al., 2024), whose data exhibit domain-specific information (Liu et al., 2024; Miao al., 2024; Zhao et al., 2023; Guo al., 2014). Information from the same or similar domain the target domain is useful for improving the model's effectiveness in the target task (Chen et al., 2023a). However, as shown in Figure (a)) existing time series pre-training frameworks (Woo et al., 2024; Liu et al., 2024; Zhou al., 2024) primarily focus on learning generalized time series representations during pre-training while overlooking Nomain--specifi representations, called direct transfer. While generalized representations are essen- tial, directly transferring them to specific downstream tasks without incorporating domain-specific information leaves room for improvement. Therefore, it is necessary to learn domain-specifie information during pre-training and adap- tively transfer the specific representations to target domain, called adaptive transfer. Realizing adaptive transfer poses two difficultics: 1) capturing domain-speciffi information in pre-training. 2) adaptive use of domain-specific information in various downstream tasks.
We propose ROSE, a novel light weight general time series forecasting model using multi-domain datasets for pre-training and improving downstream fine-tuning performance and efficiency.
We propose novel Decomposed Frequency Learning that employs multi-frequency masking to learn com- plex general temporal patterns from multi multi-domain data, empowering the model's generalization capability.
We propose a novel TS- TS-RReistter to capture domain- specific information in pre-training and enable adap- tive transfer of target-oriented specific information for downstream tasks.
To address these challenges, we propose a register assisted general time series forecasting model with decomposed fre- quency learning, namely ROSE. First, we propose Decom- posed Frequency Learning that learns generalized represen- tations to solve the issue with coupled semantic information. We decompose individual time series using the Fourier trans- form with a novel frequency-based masking method, and then convert it back to the time domain to obtain decoupled time series for reconstruction. makes complex temporal patterns disentangled, thus benefiting the model to learn gen-
Our experiments with 7 real-world benchmarks demon- strate that ROSE achieves satt--of-theaat performance in full-shot setting and achieves competitive or supe- rior results in fhw-shot setting, along with impressive transferability in zero-shot setting.
2

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
(a) Overall architecture
(b) Register in pretrain stage
(c) Register in finetune stage
Reconstruction head
Prediction
head
Xa
X
Reconstruction Decoder
copy
Prediction
Decoder
Patchily
Patchily
Register
Register
Transformer Encoder
Cluster
Select
arg min
arg Jola
Patchify
Patchify & Projection
Cluster fl
Multifreq
Projection
Projection
Projection
Input Time Series
Input Time Series
Patch token
Register token
Forward
Backward
Training
Freeze
Figure 2. The model architecture of ROSE.
2. Related Work
2023a) use a GPT-style pre-training approach, giving rise to improved performance time series prediction. However, the above methods overlook domain-specific information from multi-source data, thus limiting the performance of the models. Different from previous approaches, ROSE pre- trains large-scale data from various domains and it con- siders both generalized representations and domain information, which facilitates flexible adaptive transfer in downstream tasks.
2.1. Traditional Time Series Forecasting
The statistical time series forecasting models like ARIMA (Box & Jenkins, 1968), despite their theoretical support, are limited in modeling nonlinearity. With the rise of deep learning, many RNN-based models (Cirstea et al., 2019; Wen et al., 2017; Salinas et al., 2020) have been pro- posed, modeling the sequential data with an autoregressive process. CNN-based models (Luo & Wang, 2024; Liu al., 2022b) have also received widespread attention due to their ability to capture local features. MICN (Wang et al., 2022) utilizes TCN to capture both local and global features, while TimesNet (Wu et al., 2022) focuses on modeling 2D tempo- ral variations. However, both RNNs and CNNs struggle to capture long-term dependencies. Transformer-bas based mod- els (Zhou al., 2022; Nie et al., 2022: Wu et al., 202 la; Liu et al., 2023; Chen et al., 2024), with their attention mech- anism, can capture long dependencies and extract global information, leading to widespread applications in long- time series prediction. However, this case-by-case paradigm requires meticulous hyperparameter design for different datasets, and its predictive performance can also be affected by data scarcity.
3. Methodology
3.1. Architecture
As illustrated in Figure 2, ROSE adopts an encoder-decoder architecture for time series modeling. Its backbone com- prises multiple Transformer layers, which effectively pro- cess sequential information and capture temporal depen- dencies (Vaswani et al., 2017). Both the reconstruction decoder and prediction decoder share the same structure as the Transformer encoder and are designed for reconstruc- tion and prediction tasks, respectively. The reconstruction task enables the model to gain comprehensive understand- ing of time series, while the prediction task enhances its few-shot and zero-shot capabilities. ROSE is pre-trained in a channel-independee way. which widely used time series forecasting (Nie et al., 2022).
2.2. Time Series Forecasting Foundation Model
Pre-training with multiple sources time series has recently received widespread attention (Rasul et al., 2023; Dooley et al., 2024; Garza & Sergenthaler-Canseer marthi & Prakash, 2023). MOMENT (Goswami et al., 2024) and MOIRAI (Woo et al., 2024) adopt BERT- style pre-training approach, while Timer (Liu et al., 2024), Chronos (Ansari et al., 2024) and TimsEM (Das et al.,
Input representations. To enhance the generalization of ROSE for adaptive transferring from multi-domains to dif- ferent target domains, model the inputs with patch tokens and register tokens. Patch tokens are obtained by partitioning the time series using patching layers (Nie et al., 2022), to preserve local temporal information. Register tokens that capture -ppeiff information will be in-
2023; Ka-
3

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
troduced in Section 3.3.
components below T7 will be masked, indicating mask low frequency, exemplified by threshold 72 in Figure 3.
3.2. Decomposed Frequency Learning
After obtaining the mask matrix M, we replicate Xfreq Kr times to get the Xfreq E C and perform element- wise Hadamard product with the mask matrix M to get masked frequency of time series. Then, we use the inverse Real Fast Fourier Transform (irFFT) to convert the results from the frequency domain back to the time domain and get K2 masked sequences Xmakk = mask jim where each x mmakk IR_ corresponding to masking with different threshold Tj.
As shown in Figure 1, time series data are composed of multiple superimposed frequency components, resulting in the overlap of different temporal changes. Furthermore, low-frequency components typically contain information about overall trends and longer-scale variations, and high- frequency components usually contain information about short-term fluctuations and shorter-scale variations, there- fore, understanding time series from low and high frequen- cies separately benefits general time series representation learning. Based on the observations above, we propose novel frequency-based masked modeling that randomly mask either figh-freeeency or low- www-freuenccy components of time series multiple times as the key to enable learning of common time series patterns, such as trends and various long and short term fluctuations. Finally, reconstruction task assists the model in comprehending the data from various frequency perspectives, enabling it to learn generalized rep- resentations. In contrast, existing frequency masking meth- ods (Zhang et al., 2022), which randomly mask frequencies of a single time series once, show limited forecasting effec- tiveness due to the lack of common pattern learning from heterogeneous time series that come from various domains.
Krr
Xmakk irFFF((Xttee C M).
(3)
Representation learning. As shown in the yellow part of Figure 3, after obtaining the K2 masked sequences maskk we divide each sequence x into P' non-overlapping patches, and use linear layer to transforming them into P patch tokens, and thus we get Xmp = (X )
mp i=1 to capture general information, where each ump E PxD and D is the dimension for each patch token. We replicate the reg- ister tokens X,, K2 times to get X, E Kgx,N,,0 XD where X, is obtained by inputting the original sequence into the TS- Teg-teis,,,, as detailed in Section 3.3. Then, we concate- nate the patch tokens Xmp with the register tokens X,, and feed them into the Transformer encoder to obtain the repre- sentation of each masked series. These representations are then averaging aggregated to yield a unified representation Sm E R(N++))DD
Multi-frequency masking. As shown in the green part of Figure 3, given time series x E R", we utilize the Real Fast Fourier Transform (fFFT) (Brigham & Morrow, 1967) to transform it into the frequency domain, giving rise to Xfreq E C4/++1
S. =
Ku))) (4)
Xfteq = (x).
(L)
Reconstruction task. After obtaining the representation Sm. we feed it into the reconstruction decoder, which shares same stucture as the Tranformer encoder, and ultimately reconstruct the original sequence & E TR through the re- construction head, which a linear layer. As frequency domain masking affects the overall time series, we compute the Mean Squared Error (MSE) reconstruction loss for the entire time series.
To separately model high-frequency and low-frequency information in time series, we sample Ke thresh- olds T1, T2, T3.... TK; and Kr random numbers for multi-frequency masks, where Uniform(0,a) a). a- < L/2 + 1, and μ E Bernoulli(p). Each pair of 7; and μ4 corresponds to the ith frequency mask. This generates mask matrix M E kJ x(L/2+1), where each row corresponds to the in. frequency mask, each column corresponds to the jth frequency, and each element mij is 0 or 1, meaning that the jth frequency is masked with the ish frequency mask or not.
M1:
Creconstruction = 1|x- -x|
(5)
3.3. Time Series Register
By decomposed frequency learning, we can obtain the general representations. Additionally, we propose the TS- Register that learns domain-specific information from the multi-domain datasets for adaptive transfer. It clusters Nomain-specinn information from the multi-domain datasets into register tokens and stores such domain-specine informa- tion in the register during pre-training. Then, it adaptively selects Nomain-specifte information from the register via a Top-K selection strategy to enhance the performance in the
maj { Pi
j i .if j n
(1-
(2)
where and μ4 denote the threshold and random number for the its frequency domain mask. If μ. 1, it means that frequency components above T; will be masked, indicating to mask high frequency, , as shown by the threshold Tı in Figure 3. Conversely, if 14 = O, it signifies that frequency
4

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Time Domain
Frequency Domain
Value
Amp
Threshold
Thonsbol Treenae
Reconstructed time series
Reconstruction head
Time
Frequency
Mask
Random Mask
Multi Frequence Mask
Reconstructionn Decoder
Hi 1, Mask High Freq
lowhigh Frea
Mask
Aggregator
InFT
Mask
Mark
Transformer Encoder
Masked time series threshald THt
Masked series threshald F
Concat register tokens
D, Mask Low Freq
Ms 1, Mask High Freq
Masked threshold fz
Masked sories threshold T3
Masked time series
Figure 3.
An illustration of decomposed frequency learning. Based on the sampled thresholds, we randomly apply low/high-frequeney masking to the time series in the frequency domain and then transform it back to the time domain for reconstruction.
target domain. A novel learnable low-rank matrix is pro- posed to set to complement the downstream dataset-specific information through fine-tuning.
D is the dimensionality of Transformer latent space. X is called register tokens, which are used as the prefix of the patch tokens Xp E R and input for the Transformer encoder to provide domain-specific information.
We set up a randomly initialized register E x with H cluster center vectors e, E R... E f1. H) Each of imput time series x E R- is projected into data- dependent embedding X. E DR. through a linear layer.
Fine-tuning stage. As shown in Figure 2(c), after obtain- ing register E that contains domain-speciff information through pre-training, we freeze the register parameters to adaptively use this domain-specific information in the down- stream tasks.
Pre-training stage. As shown in Figure 2(b), we use the register to cluster these data-dependent embeddings, which generate domain-specift information, and store them in pre-training. Specifically, We find cluster center vector e, from the register E where we use å to denote the cluster that the data-dependent embedding xe belongs to.
Since the target domain may not strictly fit one of the up- stream domains, we propose novel embedding learning of the downstream data by employing a Top-K strategy that sclects k similar vectors in the register. As shown in Equa- tion 7, the embedding of input time series x. picks the k nearest vectors in the register E, and uses their average as 6g to represent the domain-spectio pre-train stage. e₁ is also patched into Xa E I NDD and is used as the domain specific register tokens.
fregister = ||xe- es 1|2:
arg min ||xe ey112- j=1:H
(6)
information from the
To update the cluster center vectors in the register E that rep- resents the domain information of the pre-trained datasets, we set the loss function shown in Equation 6 that minimizes the distance between the embedding x, and the cluster cen- ter es. To solve the problem that the gradient of the arg min function cannot be backpropagated, we use the stop gradient operation to pass the gradient of es directly X..
Σ es (ō1, ,6 arg Topk ek K
j=1-H
X
e,
Since the downstream data has its own specific information at the dataset level in addition to the domain level, this may not be fully represented by the domain information obtained from the pre-trained dataset alone. Therefore, we innovatively set learnable matrix A R adjust X. to complement the specific information of downstream data. Since the pre-trained model has a very low intrinsic dimension (Aghajanyan et al., 2020), in order to get better fine-tuning results, A is set as a low-rank matrix:
In this way, the vectors in the register E cluster the em- beddings of different data and learn the domain-specitn centers for pre-trained datasets, which can represent domain- specific information. As a vector in the register E, e₂ rep- resents the domain-specifie information for input x. eş is invariant under small perturbations in x, that represents x, which promotes better representation of domain-specinn in- formation and robustness of the vectors in the register. This also avoids their over-reliance on detailed information about specific datasets.
A=uxv"
(8)
The cluster center vector eş is then patched into X, E N.NDD, where N, is the number of the register tokens and
where u E R. and v E RD, and only the vectors u and v need to be retrained in the fine-tuning step. As illustrated
5

---

Towards General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
in Equation 9, the register token X, of the downstream see- mario is obtained by doing the Hadamard product of Xa. which represents the domain-specific information obtained at the pre-train stage, and A, which represents the down- stream dataset-specific information.
of this, we gather many publicly available datasets from various domains, including energy, nature, health, transport, web, economics, etc. The details of these datasets are shown in the Appendix A.1.1. To enhance data utilization, we downsample fine-grained datasets to coarser granularity, resulting in approximately 887 million time points.
X, = Xd O.
Evaluation datasets. To conduct comprehensive and fair comparisons for different models, we conduct experiments seven well- forecasting benchmarks as the target datasets, including Weather, Traffic, Electricity, and ETT (4 subsets), which cover multiple domains.
3.4. Training
To improve the prediction performance in zero-shot and few- shot settings, we co-train supervised prediction with self- supervised reconstruction that uses multi- multi-frequency mask- ing to learn unified features that are more applicable to the downstream prediction task.
Baselines. We select the state-of-the-ant models as base- lines in full-shot and few-shot setting, including four spe- cific models: iTransformer (Liu al., 2023), PatchTST (Nie et al., 2022), TimesNet (Wu al., 2022), and DLinear (Zeng et al., 2023), and two LLM-based models: GPT4TS (Zhou et al., 2024) and 1'I--LLM (Pan et al., 2024). In ad- dition, we select five foundation models for comparison in zero-shot setting, including Timer (Liu et al., 2024), MOIRAI (Woo et al., 2024), Chronos (Ansari et al., 2024), TimesFM (Das et al., 2023b), and Moment (Goswami et al., 2024).
Prediction task. The input time series E R is sliced into P non-overlapping patches and then mapped to X, E RPxD. Based on common forecasting needs (Qiu et al., 2024), we set up four prediction heads mapping to prediction lengths of [96, 192, 336, 720} to accomplish the prediction task. Patch tokens X, are concatenated with the register tokens X, and then successively fed into the Transformer encoder to yield the representation S E ()N++))
S = Cincode(Cooncateni
K.)).
(10)
Setup. Consistent with previous works, we adopted Mcan Squared Error (MSE) and Mean Absolute Error (MAE) as evaluation metrics. Due to ROSE mostly aims at long-term predictions, for fair comparison, all methods fix the look- back window L = 512 and predict the future values with lengths F = {96, 192, 336, 720). More implementation details are presented in the Appendix A.1.3.
We feed the representation S into the prediction decoder and prediction heads to obtain four prediction results Y F, where F {96, 192, 336, 720). With the ground truth YF, the prediction loss prediction is shown in Equation 11.
Eprediction
Σ
| Y Fl
(11)
(9, 720)
4.1. In Distribution Forecasting
Pre-training. The reconstruction task learns generalized features through the Transformer encoder and reconstruc- tion decoder. To utilize these features for the prediction task, the parameters of the reconstruction decoder are copied to the prediction decoder during forward propagation. To avoid prediction training affecting the generalization perfor- mance of the model, the gradients of the prediction heads are skipped at back- pockkpropagation. The overall loss of ROSE in pre-training stage is shown in Equation 12.
Setting. In full-shot setting, we utilize full downstream data to fine-tune pre-trained ROSE and baselines. In few-shot set- ting, fine-tune all models with only 10% train data. The "Drop Last" issue is reported by several researchers (Qiu et al., 2024; 2025a; Li et al., 2025). That is, in some previous works evaluating the model on test set with drop-last=TTrue setting may cause additional errors related to test batch size. In our experiment, to ensure fair comparison in the future, we set the drop last to False for all baselines avoid this issue.
Cprettmin Creconsrmccionn + Eprediction + Cregister
(12)
Full-shot results. As shown in Table 1, we also present the results of the ROSE in 10% few-shot setting. Key observa- tions are summarized as follows. First, as general forecast- ing model, ROSE achieves superior performance compared to the six state-of-the-art baselines with full-data training, achieving an average MSE reduction of 15%, which shows that decomposed frequency learning and register help to learn generalized representations from large-scale datasets and adaptively transfer the multi-domain information to specific downstream scenarios. Second, we observe that
Fine-tuning. We only perform a prediction task in fine- tuning. Patch tokens Xp are concatenated with the adjusted register tokens X,. For downstream task with a fixed prediction length, use the corresponding pre-trained pre- diction head to fine-tune the model.
4. Experiments
Pre-training datasets. The datasets are crucial for pre- training general time series forecasting model. In light
6

---

Towards General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
ROSE in 10% few-shot setting shockingly improves a large margin as MSE reduction in average exceeding 12% over the baselines trained with full data. This observation val- idates the transferability of ROSE pre-trained with large multi-source data.
4.3. Model Analysis
Efficiency analysis. To exhibit the performance and ef- ficiency advantages of ROSE, we compare its parameter count to other foundation models and evaluate their per- formance and testing time averaged on ETTh1 and ETTh2 datasets in zero-setting. Similarly, for each specific model, we evaluate its parameter count as well as its performance in full-shot setting and training- training-to-testing to-testing time averaged on the same datasets. Specific implementation details and results can be found in Appendix A.2.
Few-shot results. The results under the 10% few-shot set- ting are presented in Table 13 in Appendix 1.... ROSE outperforms advanced models when training data is scarce in the target domain. Figure 4 shows the performance of pre- trained ROSE and ROSE trained from scratch on ETTh1 and ETTm2 with different fine-tuning data percentages, noting the best baselines in full-shot setting. The pre-traines ROSE shows stable, superior performance even with limited fine- tuning samples. Specifically, the pre-trained ROSE exceeds SOTA performance with only 1% train data for ETThl and 2% for ETTm2. Moreover, compared to the ROSE trained from scratch, the pre-trained ROSE exhibits slower decline in prediction performance with the reduction of fine- tuning data, demonstrating the impressive generalization ability of ROSE through pre-training.
Average MSE in ETTh (2 subsets)
0.56
Maneet
0.52
Cheases 40YLITI
Avorage MSE
0.48
TInoFM 2000L1115
0.44
31EM22.9
0.40
ROSE
Timue $3.ANUS
7.4M.8655
PoichTST 1.2M.355
ETTh1
ETTm2
0.36
0.37
0.5
50
Time [Seconds)
0.42
0.27
Figure 5. Model performance, number of parameter and efficiency comparison.
0L47
As shown in Figure ROSE is a lightweight general model with 7.4M parameters and short inference time, which are only about one-tenth of the second fastest/smallest foun- dation model (Timer). Importantly, ROSE uses the least number of parameters among foundation models, with its parameter count approaching that of specific models, while exhibiting superior zero-shot performance. This is attributed to our proposed decomposed frequency learning that en- hances the comprehension of time series. Concurrently, the TS- Register achieves the adaptive transfer thus efficiently adapting to downstream tasks without the need of scaling up to achieve strong generalizability. Compared to foundation models with large scale, ROSE may better meet the need for general models in real scenarios that require high computa- tional and parameter efficiency as well as high prediction accuracy with scarce downstream data.
RosE thras Scraisk)
0.37
RUSE theme Sooral
SOTA (Palkse0
SUTAA (haliaku))
0.37
20
100
107 1%
Tine-tune Data Percentage
Fine-tune Data Percentage
Figure 4. The forecasting results of ROSE obtained by training from scratch and fine-tuning from the pre-trained model. The right, upper corner is the best case.
4.2. Zero-shot Forecasting
Setting, In this section, to ensure a fair comparison, we con- duct zero-shot predictions for each foundational model on downstream datasets not included in their pre-training data. It is worth noting that, unlike a few foundation models (Woo et al., 2024) that require much longer inputs to achieve bet- ter predictive performance, baselines to 512 without considering longer input lengths, as many real- world scenarios could offer very limited samples.
fix the input length of all
Visualization of TS-Register. To validate the TS TS-Register's s capability to transfer domain-specific information adaptively from pre-training datasets to target datasets, we visualize the cosine similarity of register vector selections from datasets across different domains. As shown in Figure 6(a), the co- sine similarity higher for datasets within the same domain and lower between different domains. We also visualize the register vector selections from different datasets in Fig- ures 6(b) and (c), where datasets from the same domain show similar visualizations. This confirms the TS- "S-Register's 's capability of adaptive transfer from multi-source to target datasets across various domains.
As shown in Table 2, ROSE significantly outper- forms across the majority of datasets, achieving an average reduction of 15% in MSE. In comparison to Timer and Moirai, ROSE achieves average MSE reductions of 9% and 6%, respectively, and demonstrates a remarkable 43% rel- ative improvement over Moment. Notably, ROSE stands out not only for its superior performance but also for its exceptionally lightweight and efficient design, which sets it apart from other foundational models. Detailed analysis of these aspects will be presented in Section 4.3.
Results.
7

---

Towards General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
Table 1. The results for ROSE in full-shot setting and 10% few-shot setting, compared with other methods in full-shot setting. The average
results of all predieted lengths are listed here.
Models
ROSE
ROSE (10%)
ITrrnsformer
PatchTST
Timesnct
Dlincar
GPT4TS
S'IP-LLM
Metric
MSR
MAF
MSE
MAR
MSR
MAF
MSB
MAE
MSE
MAE
MSE
MAR
MSE
MAR
MSE
MAE
ETThl
0.391
0.414
0.397
0.419
0.439
0.448
0.413
0.434
0.582
0.533
0.416
0,436
0.427
0.426
0.406
0.427
ETTh2
0,331
0.374
0.335
0.380
0.374
0.406
0.331
0.381
0,409
0,438
0,508
0.485
0.354
0.394
0,347
0.391
ETTml
0.341
0.367
0.349
0.372
0.362
0.391
0.353
0,382
0.490
0,464
0.356
0.378
0.352
0.383
0.343
0.379
ETTm2
0.246
0.305
0.250
0.308
0.269
0.329
0.256
0.317
0.317
0.358
0.259
0.325
0.266
0.326
0.257
0.319
Weather
0,217
0.251
0.224
0.252
0.233
0.271
0.226
0.264
0.329
0.336
0.239
0,289
0.237
0.270
0.222
0.259
Electricity
0.155
0.248
0.164
0.253
0.164
0.261
0.159
0.253
0,195
0.296
0.166
0.267
0.167
0.263
0.161
0.257
Traffic
0.390
0.264
0.418
0.278
0.397
0.282
0.391
0.264
0.623
0.333
0.433
0.305
0.414
0.294
0.405
0.286
Table 2. The results for ROSE and other foundation models in the zero-shot setting. The average results of all predicted lengths are listed here. We use '-' to indicate that the datasel has been involved the model's pre-training, and thus not used for testing.
significantly drops on E and ETTh2 with limited sam- ples. Without the reconstruction task, our model shows negative transfer effects on ETTml and ETTm2, likely due to the prediction task making the model more susceptible to pre-training data biases.
Models
ROSE|
Timer
MOIRAI
Chronos
TimesFM
Moment
Metric
MSE
MSE
MSE
MSE
MSE
MSE
Masking method. To further validate the effectiveness of decomposed frequency learning, we replace the multi- frequency masking with different masking methods, includ- ing two mainstream time-domain methods: patch mask- ing (Nie et al., 2022) and multi- masking (Dong et al., 2024), as well as random frequency masking (Chen et al., 2023b). The results in Table 4 show that random frequency masking and patch masking led to negative transfer on ETTml and ETTm2, likely due to significant disruption of the original time series, causing overfitting. In contrast, multi-patch masking and multi-frequency masking resulted in positive transfer across all datasets by preventing exces- sive disruption. Multi-frequence masking achieved better results, demonstrating its ability to help the model under- stand temporal patterns from a multi-frequency perspective. We also compare with some other pre-training tasks in Ta- ble 14 in Appendix 1.3..
ETTh1
0.401
0.451
0.475
0,560
0.489
0.708
ETTh2
0.346
0.366
0.379
0.392
0.396
0.392
ETTml
0.525
0.544
0.714
0.636
0.434
0.697
ETTm2
0.299
0.360
0.343
0.313
0.320
0.319
Weather
0.265
0.292
0.267
0.288
0.291
Electricity
0.234
0.297
0.241
0.245
0.861
Traffic
0.588
0.613
0.615
1.411
ind
KTTd
PESSA
Werberr
Ti regleee vectes wnlerii tine Drec sackes deeuko
do Thriegieer conters ielotion Ir pestrala dara
The regitie resien orlrttie Sioa targul Cata
Figure 6. Visualization of Te-Reiitter. The calculation of cosine similarity is in the Appendix A.3.
Scalability and sensitivity. The scalability analysis of ROSE's model size and pre-training data size are presented in Appendix A.4. The sensitivity analyses for the upper bound a of the thresholds, the number of masked series Kr, the number of register tokens N,, the size of register H and number of selections k in Top-K strategy are presented in Appendix A.S.
Table 3. Ablations key components of model architecture, in- cluding TS registet,, prediction task and reconstruction task. The average results of all predicted lengths are listed here.
ETTml
HTTm?
EMTh
RTT22
Design
MSF
MAR
MSB
[
MAR
MSF
MAR
MSF
MAR
ROSSE
0.39
6.372
0.250
0.308
0.397
6.419
| 0,335
0.380
TS Reginerr
0.354
0,398
0.256
|
0.312
04488
0.427
|
0.355
|
0.390
Prediction Tadk
0.300
0,384
0.257
|
0.314
0.422
0,438
|
0.372
|
0.410
Reconstruction Tack
0.357
0,403
0,269
| 0.327
0.412
0.428
|
0.361
|
0.399
seratch
0.371
0
0,261
| 0.318
0.470
0.480
|
0,400
| 0.425
4.4. Ablation Studies
Table 4. Ablations on decomposed frequency learning, where we replace Multi-freq masking with other masking methods. The average results of all predicted lengths are listed here.
Model architecture. To validate effectiveness of our model design, we perform ablation studies on TS-Register, predic- tion tasks, and reconstruction task in 10% few-shot setting. Table 3 shows the impact of each module. The TS- RS-gieisterr leverages multi-domain information during pre-training, aid- ing adaptive transfer to downstream datasets, as further discussed in Section 4.3. The prediction tasks enhance per- formance in data-scarce situations. Without it, performance
ETTml
ETTm2
ETThi
ETTR0
Design
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
ROSE
0.349
0.372
0:250
0.388
| 0.397
|
0.419
0,335
0,380
Random Freq Masking
0.381
0.397
[.26]
0.324
0.410
0.427
0.374
0.405
Molti Patch Masking
0.356
|
0.379
0:259
0.316
0.004
0.026
0.349
0.389
Putch Masking
0.398
|
0.400
| 00261
0.319
0.408
0.432
0.395
0.407
From scrath
| 0.371
| 0.301
| 0.261
0.318
0.470
0.480
0.400
0.435

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
5. Conclusion and Future Work
main adaptation for cross-domain knowledge transfer. In CIKM, pp. 234 244, 2023a.
In this work, we propose ROSE, a novel general model, ad- dressing the challenges of leveraging multi-domain datasets for enhancing downstream prediction task performance. ROSE utilizes decomposed frequency learning and TS- Register to capture generalized and domain-specific rep- resentations, enabling improved fine-tuning results, espe- cially in data-scarce scenarios. Our experiments demon- strate ROSE's superior performance over baselines with both full-data and few-data fine-tuning, as well as its im- pressive zero-shot capabilities. Future efforts will con- centrate expanding pre-training datasets and extending ROSE's applicability across diverse time series analysis tasks, e.g., classification. We provide our code at ht https: //github. weteecssiionntellleeeee... /ROSE.
Chen, M., Xu, Z., Zeng. A., and Xu, Q. Fraug: Frequency domain augmentation for time series forecasting. arXiv preprint arXiv:2302.09292 2023b.
Chen, P., Zhang. Y., Cheng, Y., Shu, Y., Wang. Y., Wen, Q.. Yang, B., and Guo, C. Pathformer: Multi-scale transform- ers with adaptive pathways for time series forecasting. 2024.
Cirstea, R.-G., Yang. B., and Guo, C. Graph attention recur- rent neural networks for correlated time series forecasting. In SIGKDD, 2019.
Das, A., Kong, W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting. arXiv preprint arXiv:2310. 10688, 2023a.
Impact Statement
This paper presents work whose goal is advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here.
Das, A., Kong. W., Sen, R., and Zhou, Y. A decoder- only foundation model for time-series forecasting, arXiv preprint arXiv:2310. 10688, 2023b.
Dau, H. A., Bagnall, A., Kamgar, K., Yeh, C.-C. M., Zhu, Y., Gharghabi, S., Ratanamahatana, C. A., and Keogh, E. The uer time series archive /EEE/CAA Journal of
Acknowledgements
This work was partially supported by National Natural Sci- ence Foundation of China (62372179). Chenjuan Guo is the corresponding author of the work.
Automatica Sinica, 6(6):1293-130) 2019.
Dong, J., Wu, H., Zhang, H., Zhang, L., Wang, J., and Long, M. Simmtm: A simple pre-training framework for masked time-series modeling. NeurlPS, 36, 2024.
References
Dooley, S., Khurana, G. S., Mohapatra, C., Naidu, S. V., and
Aghajanyan, A., Zettlemoyer, L., and Gupta, S. Intrin- sic dimensionality explains the effectiveness of language model fine-tuning, arXiv preprint 1331v:2012.13255, 2020.
White, C. Forecastpfn: Synthetically-trained zero-shot forecasting. NeurlPS, 36, 2024.
Garza, A. and Mangenthaaler-Canseoo M. Timegpt-1, arXiv preprint arXiv:2310.03588) 2023.
Ansari, A. F., Stella, L., Turkmen, C., Zhang, X., Mercado, P. Shen, H., Shchur, O., Rangapuram, S. S., Arango, P., Kapoor, S., et al. Chronos: Learning the language of time series. arXiv preprint arXiv:2403.077 078115, 2024.
Godahewa, R., Bergmeir, C., Webb, G. I., Hyndman, R. J., and Montero-Manno, Monash time series forecasting archive. arXiv preprini 1.Xiv:2105.066 2021.
Bagnall, A., Dau, H. A., Lines, J., Flynn, M., Large, J., Bostrom, A., Southam, P., and Keogh, F. The uea mul- tivariate time series classification archive, 2018. arXiv preprint arXiv: 1.0005,, 2018
Goswami, M., Szafer, K., Choudhry, A., Cai, Y., Li, S., and Dubrawski. A. Moment: A family of open time-series foundation models. arXiv preprint arXiv:2402.03885. 2024.
Box. G. E. and Jenkins, G. M. Some recent advances in forecasting and control. Journal of the Royal Statistical Society, Series C (Applied Statistics), 17(2)91-109, 1968.
Guo, C., Jensen, C. S., and Yang, B. Towards total traffic awareness. SIGMOD Record, 43(3):18-22 2014.
Kamarthi, H. and Prakash, B. A. Large pre-trained time series models for cross-domain time series analysis tasks. arXiv preprint arXiv:2311.1141. 2023.
Brigham, E. O. and Morrow, R. The fast fourier transform. IEEE spectrum, 4(12)633700, 1967.
Chen, L., Wang, L., Xu, J.. Chen, S., Wang, W., Zhao, W., Li, Q., and Wang, L.. Knowledge-inspires subdo-
Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 14l: 69800 2014.

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference Machine Learning (ICML 2000), pp. 1207-1216, Stan- ford, CA, 2000. Morgan Kaufmann.
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z.., Gimelshein, N., Antiga, L., et al. Pytorch: An imperative style, high-performance deep learning library. NeurlPS, 32, 2019.
Lee, S., Park, T., and Lee, K. Learning to embed time series patches independently. arXiv preprint arXiv:2312.16427, 2023.
Qiu, X., Hu, J., Zhou, L.,, Wu, X., Du, J., Zhang, B., Guo, C., Zhou, A., Jensen, C. S., Sheng. Z., ct al. Tib: Towards comprehensive and fair benchmarking of time series fore- casting methods. arXiv preprini arXiv:2403 Trxi::2433.20100 2024.
Li, Z., Qiu, X., Chen, P., Wang, Y., Cheng, H., Shu, Y., Hu, J., Guo, C., Zhou, A., et al. Tsfm-bench: Comprehensive and unified benchmarking of foundation models for time series forecasting. In SIGKDD, 2025.
Qiu, X., Li, Z., Qiu, W., Hu, S., Zhou, L., Wu, X., Li, Z.,, Guo, C., Zhou, A., Sheng, Z., Hu, J., Jensen, C. S., and Yang, B. Tab: Unified benchmarking of time series anomaly detection methods. In Proc. VLDB Endow., 2025a.
Liu, M.. Zeng, A., Chen, M., Xu, Z., Lai, Q., Ma, L., and Xu, Q. Scinet: Time series modeling and forecasting with sample convolution and interaction. Advances in Neural Information Processing Systems, 35:5816 51-8888, 2022a.
Qiu, X., Wu, X., Lin, Y., Guo, C., Hu, J., and Yang. B. Duet: Dual clustering enhanced multivariate time series forecasting. In SIGKDD, pp. 1185-1196, 2025b.
L Liu, M., Zeng, A., Chen, M., Xu, Z., Lai, Q., Ma, I.., and Xu, Q Scinet: Time series modeling and forecasting with sample convolution and interaction. NeurlPS, 35: 5816-5828, 2022b.
Rasul, K., Ashok, A., Williams, A. R., Khorasani, A., Adamopoulos, G., Bhagwatkar, R., Bilok, M., Ghonia, H., Hassen, N. V., Schneider, A., et al. Lag-Ilama: Towards foundation models for time series forecasting. arXiv preprint arXiv:2310.08272 2023.
Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., and Long, M. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.0662 2023.
Salinas, D., Flunkert, V., Gasthaus, J., and Januschowski, T. Deepar: Probabilistic forecasting with autoregressive recurrent networks. International journal of forecasting, 36(3):1181-1199) 2020.
Liu, Y., Zhang, H., Li, C.,, Huang, X., Wang, J., and Long, M. Timer: Transformers for time series analysis scale. arXiv preprint ar arXiv:2402.02336, 2024.
Taieb, S. B., Bontempi, G., Atiya, A. F., and Sorjamaa, A. review and comparison of strategies for multi-step ahead time series forecasting based the nn5 forecasting competition. Expert systems with applications, 39(8): 7067-7083, 2012.
Luo, D. and Wang, X. Modernten: modern pure convolu- tion structure for general time series analysis. In ICLR, 2024.
Makridakis, S. M4 dataset, 2018. https://githin com/M4Compar it cn/MM--methods/tree master/Datased
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. NeurlPS, 30, 2017.
McCracken, M. W. and Ng. S. Fred-md: A monthly database for macroeconomic research. Journal of Busi- ness & Economic Statistics, 34(4):574-555 2016.
Wang, H., Peng, J., Huang, F., Wang. J., Chen, J., and Xiao, Y. Mien: Multi-scale local and global context modeling for long-term series forecasting. In ICLR, 2022.
Miao, H., Zhao, Y., Guo, C., Yang. B., Zheng, K., Huang, F., Xie, J.. and Jensen, C. S. A unified replay-based contin- uous learning framework for spatio-temporal prediction on streaming data. In ICDE, pp. 1050-1062, 2024.
Wang, Y., Han, Y., Wang, H., and Zhang, X. Contrast every- thing: A hierarchical contrastive framework for medical time-series. NeurIPS, 36, 2024.
Nie, Y., Nguyen, N. H., Sinthong, P., and Kalagnanam, A time series is worth 64 words: Long-term forecasting with transformers. arXiv preprint arXhv:2211 2022.
Wen, R., Torkkola, K., Narayanaswamy, B., and Madeka, D. A mmulti-horizze quantile recurrent forecaster. arXiv preprint arXiv:1711..I1 2017.
Pan, Z., Jiang, Y., Garg, S., Schneider, A., Nevmyvaka, Y., and Song, D. 2 ip-lim: Semantic space informed prompt learning with Ilm for time series forecasting. In ICML, 2024.
Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., and Sahoo, D. Unified training of universal time series fore- casting transformers, arXiv preprint arXiv:2402. 02592, 2024.
10

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Wu, H., J., Wang, J., and Long, M. Autoformer: Decom- position transformers with auto-correlation for long-term series forecasting. NeurIPS, 34:22419-22453 2021a.
Zhao, K., Guo, C., Cheng, Y., Han, P., Zhang, M., and Yang. B. Multiple time series forecasting with dynamic graph modeling, Proc. VLDB Endow, 17(4):753-765 2023.
Wu, H., Hu, T., Liu, Y., Zhou, H., Wang, J., and Long, M. Timesnet: Temporal 2d- variation modeling for general time series analysis. In ICLR, 2022.
Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L.., and Jin, Fedformer: Frequency enhanced decomposed trans- former for long-term series forecasting. In ICMI., pp. 27268-27288 2022.
Wu, X., Zhang, D., Guo, C., He, C., Yang, B., and Jensen, C. S. Autocts: Automated correlated time series forecast- ing. Proc. VLDB Endow., 15(4):971-983 2021b.
Thou, T., Niu, P., Sun, I.., Jin, R., et al. One fits all: Power general time series analysis by pretrained Im. NeurIPS, 36, 2024.
Wu, X., Wu, X., Yang, B., Zhou, L., Guo, C., Qiu, X., Hu, J., Sheng. Z., and Jensen, C. S. Autocts++: zero-shot joint neural architecture and hyperparameter search for correlated time series forecasting. The VLDB Journal, (5):1743-1770, 2024.
Wu, X., Qiu, X., Gao, H., Hu, J., Yang, B., and Guo, C. K2vae: A koopman-kalln kalman enhanced variational autoen- coder for probabilistic time series forecasting. In ICML, 2025a.
Wu, X., Qiu, X., Li, Z., Wang, Y., Hu, J., Guo, C., Xiong, H., and Yang, B. Catch: Channel-aware multivariate time series anomaly detection via frequency patching. In ICLR, 2025b.
Wu, X., Wu, X., Zhang, D., Zhang, M., Guo, C., Yang, B., and Jensen, C. S. Fully automated correlated time series forecasting in minutes. In Proc. VLDB Endow., volume 18, pp. 144-157, 2025c.
Zeng, A., Chen, M., Zhang, L., and Xu, Q. Are transformers effective for time series forecasting? In AAAI, volume 37, pp. 11121-11128. 2023.
Zhang, S., Guo, B., Dong, A., He, J., Xu, Z., and Chen, S. Cautionary tales on air-quality improvement bei- jing, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473((205):2017045)) 2017.
Zhang, X., Zhao, 7.,, Tsiligkaridis, T., and Zitnik, M. Self- supervised contrastive pre-training for time series via fime-freyuency consistency. NearIPS, 35:3988-4003, 2022.
Zhang, X., Chowdhury, R. R., Shang, J., Gupta, R., and Hong, D. Towards diverse and coherent augmentation for time-series forecasting. In ICASSP, pp. 1-5. IEEE, 2023.
Zhang, X., Zhao, S., Song, Z., Guo, H., Zhang, J., Zheng, C., and Qiang, W. Not all frequencies are created equal: Towards a dynamic fusion of frequencies time-series forecasting. In ACM Multimedia, 2024.

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A. Appendix
A.1. Implementation Details
A.1.1. PRE-TRAINING DATASETS
We use multi-source datasets in pre-training which contain subsets of Monash (Godahewa et al., 2021), UEA (Bagnall et al., 2018) and UCR (Dau et al., 2019) time series datasets, as well as some other time series classical datasets (Zhang et al., 2017; Wang al., 2024; Liu al., 2022a; McCracken Ng. 2016; Taieb et al., 2012). The final list of all pre-training datasets is shown in Table There is no overlap between the pre-training datasets and the target datasets. It is worth noting that the dataset weather in the pre-traine pre-training dataset is univariate dataset, which is different to the multivariate dataset weather in the target task. The pre-trained datasets can be categorized into 6 different domains according to their sources: Energy, Nature, Health, Transport, and Web. The sampling frequencies of the datasets show a remarkable diversity, ranging from millisecond samples to monthly samples, which reflects the diverse application scenarios and complexity of the real world. For all pre-training datasets, we split them into univariate sequences and train them a Ghannel-independee manner.
Table 5.
List of pretraining datasets.
Domain
Dataset
Prequency
Time Pionts
Source
Aus. Electricity Demand
Half Hourly
1155264
Monash al., 2021)
Wind
Seconds
7397147
Monash(Godaahewa al., 2021)
Energy
Wind Farms
Minutely
172178060
Monash(Godaaewa et al., 2021)
Solar
10 Minutes
7200720
Monash(Godaaowa et al., 2021)
Solar Power
4 Seconds
7397222
Monash(Goodahewa et al., 2021)
London Smart Meters
Half Hourly
166527216
Monash(Goodahewa al., 2021)
Phoneme
2160640
UCR(Dau et al.. 2019)
LigenWWorms Worns
27947136
UEA(Bagmall et al., 2018)
PRSA
Hourly
4628448
(Zhang et al., 2017)
Temperature Rain
Daily
23252200
Monush(Godaaeewa ul., 2021)
Starl StarrlightCurrvss
9457664
UCR(Dau al., 2019)
Worms
0.033 Seconds
232200
UCR(Dau et al.., 2019)
Nature
Saugeen River Flow
Daily
23741
Monush(Godahhewa et al., 2021)
Sunspot
Daily
73924
Monash(Godaahewa el al., 2021)
Weather
Daily
43032000
Monash(Godaaewa al., 2021)
KDD Cup 2018
Daily
2942364
Monash(Goxaahewa al., 2021)
Births
Daily
7305
Monash(Godaaeew et al., 2021)
Motorlmagery
0.001 Seconds
72576000
UEA(Bagnall al., 2018)
SclfRegulationSCPI
0.004 Seconds
3015936
UEA(Bagnall al., 2018)
SelfRegulationSCCP
0.004 Seconds
3064320
UEA(Bagnall 2018)
AtriulFibrillatioon
0.008 Seconds
38400
UEA(Bagmall al., 2018)
Health
PigAntrressaree
624000
UCR(Dau et al., 2019)
PIGCVP
624000
UCR(Dau et al., 2019)
TDbrain
0.002 Seconds
79232703
(Wang al., 2024)
Pems03
Minute
9382464
(Liu al., 2022a)
Pems04
Minute
5216544
(Liu et al., 2022a)
Pems07
5 Minute
24921792
(Liu et al., 2022a)
Transport
Pems08
5 Minute
3035520
(Liu et al.. 2022a)
Pems-bay
5 Minute
16937700
(Liu al., 2022a)
Pedestrian Counts
Hourly
3133234
Monash(Godahewa al., 2021)
Web
Web Traffic
Daily
116485589
Monush(Goodahewa et ul., 2021)
FRED_MD
Monthly
77896
(McCracken & Ng. 2016)
Economic
Bitcoin
Daily
75364
Monash(Godahewa al., 2021)
NNS
Daily
87801
(Taieb al., 2012)
12

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A. 1.2. EVALUATION DATASETS
We use the following multivariate time-series datasets for downstream fine-tuning and forecasting: ETT datasets' contain 7 variates collected from two different electric transformers from July 2016 to July 2018. It consists of four subsets, of which ETTh//ETTh2 recorded hourly and ETTmm/ETTTT are recorded every 15 minutes. Traffic² contains road occupancy rates measured by 862 sensors on freeways in the San Francisco Bay Area from 2015 to 2016, recorded hourly. Weather collects 21 meteorological indicators, such as temperature and barometric pressure, for Germany in 2020, recorded every 10 minutes. Electricity* contains the electricity consumption of 321 customers from July 2016 to July 2019, recorded hourly. We split each evaluation dataset into train srinn-validation-tes sets and detailed statistics of evaluation datasets are shown in Table
Table 6. The statistics of evaluation datasets.
Dataset
ETTml
ETTm2
ETTh1
ETTh2
Traffic
Weather
Electricity
Variables
7
7
7
862
21
321
Timestamps
69680
69680
17420
17420
17544
52696
26304
Split Ratio
6:2:2
6:2:2
6:2:2
6:2:2
7:1:2
7:1:2
7:1:2
A. 1.3. SETTING
We implemented ROSE in PyTorch (Paszke et al., 2019) and all the experiments were conducted on NVIDIA A800 80GB GPU. We used ADAM (Kingma& & Ba, 2014) with an initial learning rate of x 10 and implemented learning rate decay using the StepI.R method to implement learning rate decaying pre-training. By default, ROSE contains 3 encoder layers and 3 decoder layers with head number of 16 and the dimension of latent space D = 256. The patch size for patching is set to 64.
Pre-training. We use N, = 3 as the number of register tokens and P = as the path tokens. We set the input length to 512 for the supervised prediction task with target lengths of 96, 192, 336, and 720. We also set the input length to 512 and mask number K+- 4. The batch size is set to 8192 in pre-training.
Fine-tuning. We fix the lookback window to 512, and perform predictions with target lengths of 96, 192, 336, and 720, respectively. The number of register tokens Nr and patch tokens P is the same as in pre- gre-training, and the parameter l = 3 in TopK is set when selection vectors are performed in the register.
The I-SNE visualization. We select three datasets (Pems08, PSRA, Electricity) from transport, nature and energy domains respectively and compare the differences in hidden representations between direct transfer and adaptive transfer. Specifically, direct transfer refers the case where domain specific information is not considered, while adaptive transfer considers domain specific information that is learned by register tokens. We visualized the output of the encoder's hidden representations using I-SNE.
1.1.4. BASELINES
We select the state-of-the-ar models as our baselines in full-shot and few-shot setting, including four specific models: iTransformer (Liu et al., 2023), PatchTST (Nic et al., 2022), TimesNet (Wu et al., 2022), and DLinear (Zeng et al., 2023), and two LLM--ased models: GPT4TS (Zhou al., 2024) and I'IP-LLM (Pan et al., 2024). In addition, we selected five foundation models for comparison in zero-shot setting, including Timer (Liu et al., 2024), MOIRAI (Woo al., 2024), Chronos (Ansari al., 2024), TimesFM (Das et al., 2023b) and Moment (Goswami et al., 2024). The zero-shot experiment of Moment is designed based the reconstruction task in the pre-training phase. Specifically, to ensure consistency between pre-trate pre-training (reconstruction) and downstream prediction tasks, we actively mask the time periods to be predicted in the input sequence, and directly use the model's 's reconstruction output for this part as the prediction value. Moment itself is not designed for zero-shot prediction and does not officially support zero-shot forecasting in this manner. The specific code base for these models is listed in Table 7:
httpps///gthub..co//ouuaaaa Eattatt
e https://pems.dot..u.g
https:/wwww.hg-jennmmppp
jtw.phggeein..mmgg.deeeeeeee// de/wetter/
'htpp://archive.ii ics uci. www.e//wwwwddeeeeeeeeeeeeeeaaaan
13

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Table 7. Code repositories for baselines.
Model Types
Models
Code Reposituries
iTransformer
PatchTST
Small Model
TimesNet
http://gghhuh.coommm/m////eeeee
Dlinear
Timer
http://ghhub...ommmmmmmm///////s Model
MOIRAI
hutp:://ghhaa.ccom//cccalaaaaaan
Chronos
Foundation Model
TimesFM
Moment
GPT4TS
LLM-based Model
S2IP-ILM
http://ghhhun.ccmmm/mmrjjjjssssp
A.2. Efficiency Analysis
As important aspect of foundation models, inference efficiency is crucial. Therefore, we evaluate the testing time of ROSE and five foundation models in the ETTh1 and ETTh2 dataset in zero-shot setting. Similarly, evaluate the time of the entire process of training, validation, and testing for four specific models in the same datasets in full-shot setting. The above experiments all set the batch size to 32. The specific results are shown in Table 8 and Figure 5. We observe that ROSE maintains its advantage in zero-shot performance while also being significantly faster compared to the baselines, even being approximately ten times faster than the second-fastes, foundation model, Timer. This raises our reflection on whether time-series foundation models require extremely large parameter sizes and whether existing time-series foundation models have validated their architectures' scaling laws on time-series data.
Table 8. Efficiency analysis.
Model
Parameters
Pre-train datasize
Averaged time
ROSE
7.4M
0.89B
0.652s
MOIRAT
311M
27B
7.920s
Timer
67.4M
1B
5.989s
Chronos
46M
84B
176s
TimesFM
200M
100B
10.5s
Moment
385M
1.13B
13s
Itransformer
3.8M
34.18s
PatchTST
3.2M
35.47s
TimesNet
1.8M
146s
Dlinear
0.018M
24.06s
A.3. Calculation of Cosine Similarity
In Figure 6, we visualize the cosine similarity of register vector selections. For each sample in dataset, during the inference process, l vectors are selected from the register based on the Top-K strategy. We iterate through all samples in the dataset and count the number of times each vector is selected, which allows us to obtain a record vector of length equivalent to the size of the register for each dataset. The ins position in the record vector represents the number of times the iza vector in the register has been selected by samples in the dataset. For a pair of datasets, we can obtain a unique record vector for cach dataset, and then we are able calculate the cosine similarity of two vectors.
A.4. Scalability
Scalability is crucial for a general model, enabling significant performance improvements by expanding pre-training data and model sizes. To investigate the scalability of ROSE, we increased both the model size and dataset size and evaluated its
14

---

Towards General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
predictive performance on four ETT datasets.
(a) Fine-tune 10% Samples
(b) Fine-tune 5% Samples
(c) Fine-tune 10% Samples
(d) Fine-ture 5% Samples
0.380
0.380
0.355
0.370
0,744 (2N)
1,3822 (2M)
0.3150
0.370
0:3618
0.370
0,3653
0.350
(178M)
0,363
AM)
C2 (RM)
0.360
0.360
0.345
0.3917
0.356
0.3001
0.350
IL.396
0.350
(3.2M)
0.3471
0.3100
0.310
(3130
(706M)
0.3157
0.389
(354M)
0,3155
0,300
(3.2M)
(1,8) IL3347
(4530)
(1 (MM) a 3354
(1.5M)
0.335
200
(331M)
(70880) (B1)()
0.330
0.330
0.330
0.335
2
4
10
2
8
10
20
40
60
80
100
20
40
60
80
100
Layer Number
Layer Number
Pre-train Data Percentage
Pre-train Data Percentage
Figure 7. (a): Larger ROSE demonstrates better performance on downstream forecasting, (c)(d): ROSE pre-trained on larger datasets
demonstrates better performance downstream forecasting.
Model size. Constrained by computational resources, we use 40% pre-training datasets. The results are shown in Figure 7(a) and (b). When maintaining the model dimension, we increased the model layers, increasing model parameters from 2M to 4.5M. This led to 10.37% and 9.34% improvements in the few-shot scenario with 5% and 10% downstream data, respectively.
Data size. When keeping the model size, we increase the size of the pre-training datasets from 178M to 887M The results are shown in Figure 7(c) and (d). The performance of our model steadily improves with the increase in dataset size and achieves improvements of 7.4% and 4.8% respectively.
A.5. Sensitivity
We perform the sensitivity analyses for the upper bound a of the thresholds, the number of masked series K₁, the number of register tokens N,, the size of register H and the number of selections k in Top-K strategy. All the sensitivity experiments present the average results on the four ETT datasets: ETTh1, ETTh2, ETTml and ETTm2 under 10% few-shot setting.
Number of masked series. As described in Section 3.2, we propose decomposed frequency learning, which employs multiple thresholds to randomly mask high and low frequencies in the frequency domain, thereby decomposing the original time series into multiple frequency components. This allows the model to understand the time series from multiple frequency perspectives. In this experiment, we study the influence of the number of masked series K2 on downstream performance. We train ROSE with 2, 3, 4, 5, or 6 mask series. We report the results of this analysis in Figure 8(a). We find that as the number of masked sequences increases, the downstream performance gradually improves. This is because the model can better understand the time series from the decomposed frequency components, which enhances the model's generalization ability. However, more masked series do not bring better downstream performance. This could be due to an excessive number of masked sequences leading to information redundancy. In all our experiments, we keep 4 mask series.
Number of register tokens. The TS-register module presented in Section 3.3 supports the configuration of an arbitrary number of register tokens. In Figure 8(b), we visualize the relationship between the performance on the ETT datasets under 10% few-shot setting and the number of register tokens. is observed that when the number of register tokens ranges from 1 to 6, the model's performance remains relatively stable, with an optimal outcome achieved when the number is set 3. This phenomenon may be because when the number of register tokens is too small, they contain insufficient domain-specific information, which limits their effectiveness in enhancing the model's performance. Conversely, an excess of register tokens may introduce redundant information, hindering the accurate representation of domain-specific information. Additionally, we compared the results without the adjustment of a low-rank matrix on the register tokens and found that the incorporation of a low-rank matrix adjustment led to improvements across all quantities of register tokens. This finding underscores the significance of utilizing low-rank matrix to supplement the register tokens with downstream data-specific information.
Thresholds upper bound. Figure 9()) illustrates the relationship between threshold upper bound and model performance. We have observed that the upper bound of the threshold has a minimal impact on the model's performance. Generally, the information density is higher in low-frequect wwwffreueeccy components compared to high-frequency ones. Therefore, the upper bound of the threshold should be biased towards the low-frequency range to balance the information content between ow-frequency and figh-freguency components. However, this bias should not be excessive. Our experiments indicate that an upper bound of T./10 performs worse than I./5 as an overly left-skewed threshold results in insufficient information in the foe-freyy
15

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Fine-tune 10% Samples
Fine-tume 10% Samples
0,342
0.345
w/o low-rank matrix
with Imw-rank matrix
0.339
0.3377
0.340
0.3382
0,3395
0.3101
MSE
0.3330
MSE
0.3053
II 0.3330
0.2368
0:3172
0.3336
+0.3347 0.3349
0.333
0.3328
0.30322
0.335
0.3300
0.330
0.330
2
6
Mask Number
Register Token Number
(a)
(b)
Figure 8. (a): Analysis of the number of masked series. (b): Analysis of the number of register tokens.
range, making the reconstruction task either too difficult or too simple. Based on our findings, we recommend using L/5 as the upper bound for the threshold.
Register size. Figure 9(b) illustrates the relationship between register size and model performance. The register size determines the upper limit of domain-speciffe information that the register can store. We can observe that there is a significant improvement in the model effect when the register size is increased from 32 to 128. When the register size exceeds 128, the improvement the model effect with the increase register size no longer obvious. Therefore, we believe that 128 an appropriate register size for the current pre-training datasets.
Number selections in Top-K strategy. Figure 9(c) illustrates the relationship between the number of selections it in the register realize adaptive transfer of domain-specifie information
Top-K K strategy and model performance when in downstream tasks. It can be seen that the model effect performance peaks at 3 tokens at & = which has some advantages over selecting once (A: = 1), indicating that the TopK strategy can compensate for the problem of incomplete matching of upstream and downstream domains to some extent. However, too large k will also introduce redundant information and limit the accuracy of comain-specifii information transfer. Fine- tune on 10% Samples
Fine-tune on 10% Samples
Fine-tune on 10% Samples
0.350
0.350
0.350
0.361
0.340
0.337
0.335
0.335
6.330
0.335
0.335
0:3307
0.335
0,333
0.325
0.330
0.330
0.330
0.325
0.325
64
Number of selections
Threshold upper bound
Register size
in Top-K strategy
(a)
(b)
(c)
Figure 9. (a): Analysis of the threshold upper bound. (b): Analysis of the size of register. (c): Analysis of the number of selections in Top-K strategy.
A.6. Short-term Forecasting
We also try to apply ROSE to short-term forecasting on the M4 (Makridakis, 2018) dataset, which contains the yearly, quarterly and monthly collected univariate marketing data. We follow TimesNet's (Wu et al., 2022) setting and metrics (SMAPE, MASE and OWA) for testing. As shown in Table 9, ROSE also exhibits competitive performance the M4 dataset compared to the baselines.
Table 9, Full results on Short-term forecasting.
Thundem
PuchTT
Diince
GPPT4TS
SHINLLL
SMAPE
OWA
SMAPE
SMAPE
OW/
SMAPE
OWN
SMAPE
0.823
3.015
Quntizly
8.885
10.00
01940
12.152
0.965
1.520
Othrn
4.688
3.166
1,020
6.588
4.11
1.401
6.067
1.417
1005
6.709
4883
1,487
3,228
3.242
16

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A.7. Model Generality
We evaluate the effectiveness of our proposed multi-frequency masking on Transformer-based models and CNN-based models, whose results are shown in Table 10. It is notable that multi-frequency masking consistently improves these forecasting models. Specifically,, it achieves average improvements of 6.3%, 3.7%, 1.5% in Autoformer (Wu al., 202 TimesNet (Wu et al., 2022), and PatchTST (Nie et al., 2022), respectively. This indicates that multi- multi-frreuenny Masking can be widely utilized across various time series forecasting models to learn generalized time series representations and improve prediction accuracy.
Table 10. Performance multi frqqency masking.
Datasets
ETTml
ETTm2
ETTh1
ETTh2
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
Autoformer
0.600
0.521
0.328
0.365
0.493
0.487
0.452
0.458
Muutt-frequeecy Masking
0.549
0.488
0.306
0.349
0.474
0.478
0.406
0.425
TimesNet
0.400
0.406
0.291
0.333
0.458
0.450
0.414
0.427
+Multi-frqqucncy Masking
0.386
0.398
0.282
0.324
0.446
0.438
0.386
0.403
PatchTST
0.353
0.382
0.256
0.317
0.413
0.434
0.331
0.381
*Multi-frquuenc Masking
0.347
0.372
0.252
0.308
0.405
0.424
0.337
0.379
A.8. Results Deviation
We have conducted ROSE three times with different random seeds and have recorded the standard deviations for both the full-shot setting and the 10% few-shot setting, illustrated in Table 11. As the baselines didn't ' report deviations in the original paper, we only reported the deviations of the PatchTST in the full-shot setting as a comparison. It can be observed that ROSE exhibits stable performance.
Table 11. Results deviation.
Models
ROSE
ROSE (10%)
PatchTST
confidence interval
Metric
MSE
MAE
MSE
MAE
MSE
MAE
ETTml
0.342:10.000
0.367=0.002
0.349-10.003
0.372=10.002
0.349: 0.004
0.383 003
99%
ETTm2
0.246:10.000 0.02
0.303+00004
0.249=100002
0.308:10.002
0.255=10002
0.31410.000
99%
ETTh1
0.392:t0.000
0.413=0.004
0.397=00003
0.409=00003
0.411=0.003
0.432
99%
ETTh2
0.330:t0.008
0.374=0.002
0.335=+0.004
0.380=
0.348=00004
0.390 004
99%
Traffic
0.391++0.000
0.266=0.005
0.418=0.011
0.278=
0.404=0.009
0.283
99%
Weather
0.217410.008
0.250-1.0007
0.20470.007
0.252-10.009
0.223
0.263100004
99%
Electricity
0.156.10.007
0.249
0.164410.004
0.253-10.004
0.163
0.261100013
99%
17

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A.9. Visualization
A.9.1. VISUALIZATION ANALYSIS
To showcase the benefits cross-domain pre-training, we performed visualizations in both the zero-shot setting and full-shot setting.
Zero-shot: We pre-train the baselines iTrrasformee and PatchTST on the energy domain dataset ETTml and test their zero-shot performance on two different domains (weather, traffic). ROSE, without fine-tuning, is evaluated to the same two lest-sets. As shown in the Figure 10, we find that the baselines generally perform worse during domain shift due to their poor generalization. However, ROSE excels in scenarios across all domains, which demonstrates the benefits of cross-domain pre-training for improving generalization.
Full-shot: We train the baselines the train-set of downstream dataset ETTh2 and fine-tune ROSE on the same train-set. As shown in the Figure 11. We find that the baselines is limited by data diversity, leading poor performance patterns which rarely appear. However, ROSE excels in these cases, as the cross-domain pre-training allows ROSE to learn diverse temporal patterns, and helps ROSE to predict the patterns which rarely appear in the downstream train-set well.
GouedFuth
130 12
180
16
38
ROSE Weather
iTransformer Weather
PatchTST Weather
36-
2.5
15
06
60
-12
-2.
ROSE Traffic
ITranforrmr Traffic
ST Traffic
Figure 10. Visualization comparison of ROSE with crose-domain pre-training and other SOTA haselines in zero-shot setting three
domain datasets.
A.9.2. VISUALIZATION SHOWCASE
To provide distinct comparison among different models, we present visualizations of the forecasting results on the ETTh2 dataset and the weather dataset in different settings, as shown in Figures 12 to Figures 15, given by the following models: DLinear (Zeng et al., 2023), TimesNet (Wu et al., 2022), iTransfomrer (Liu et al., 2023), and PatchTST (Nic et al., 2022). Among the methods, ROSE demonstrates the most accurate prediction ability.
18

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Ehui Frecciin
200
ROSE_ ROSE_EITh2 EITh2 RarePatterr
iTransfomer ETTh2 RarePattern
ST ETTh2 RarePattern
1.80
0.0
0.0
0.4
-025
6.-
-000
4.5
4.2
300
315
156
125
ROSE ETTh2 CommonPattern
ITransformer TT CommonPattern
PatchTST ETTh2 CommonPattera
Figure Visualization comparison ROSE with cross-domain pre-training other SOTA baselines the full-shot setting rare and common patterns.
-30
-40
-22-
Etronsfomer
Patch EST
Figure 12, Visualization of input-512 and predict-336 forecasting results on the ETTh2 dataset full-shot setting.
19

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
40
ROSE
Diinear
TimesNet
200
Itransfomer
PatchTST
Figure 13. Visualization of input-512 and predict-336 forecasting results on the ETTh2 dataset in 10% few-shot setting.
-33
-20
ROSE
TimesNet
-20
-10
-1.
-54
Trransfomer
PatchTST
Figure Visualization input-512 and predict-336 forecasting results on weather dataset full-shot setting.
-1.
-12
-14
-1.
-14
TimesNet
-40
-12
PatchTST
Figure 15. Visualization of input-512 and predict-336 forecasting results the weather dataset in 10% few-shot setting.
20

---

Towards General Time Series Forecasting Model with Unified Representation and Adaptive Transfer
A.10. Full Results
A.10.II FULL-SHOOT RESULTS
Table 12 shows the full results of ROSE in full-shot setting for four prediction lengths, ROSE shows the advantage over the specific models and LLM-based models trained with the full training set.
Table 12. Full results in full-shot setting.
Models
RewdyTS
ITTranformme
PatchTST
Timesnet
Dlinear
GPT4TS
S'IP-LLM
Metric
MSE
MAE
MSR
MAR
MSF
MAE
MSB
MAF
MSF
MAE
MSR
MAE
MSR
MAR
96
0.354
0.385
0,386
0.405
0.370
0,400
0.470
0.470
0.367
0.396
0.376
0.397
0,366
0.396
192
0.389
0.407
0.424
0.440
0.413
0.429
0.568
0.523
0.400
0.417
0.416
0.418
0.401
0.420
ETTh1
336
0.406
0.422
0.449
0,460
0.422
0,440
0.595
0.547
0.428
0.439
0.442
0.433
0.412
0.431
720
0.413
0.443
0.495
0.487
0.447
0,468
0.694
0.591
0.468
0.491
0.477
0.456
0.440
0.458
avg
0.391
0.414
0.439
0,448
0.413
0,434
0.582
0.533
0.416
0.436
0.427
0.426
0.406
0.427
96
0.265
0,320
0.297
0.348
0.274
0.337
0.351
0.399
0.302
0.368
0.285
0.342
0.278
0.340
192
0.328
0.369
0.371
0.403
0,341
0.382
0.394
0.429
0.404
0.433
0.354
0.389
0.246
0.385
336
0.353
0.391
0.404
0.428
0,329
0.384
0.415
0.443
0.511
0.498
0.373
0.407
0.367
0.406
ETTM2
720
0.376
0.417
0.424
0.444
0.379
0.422
0.477
0.481
0.815
0.640
0.406
0.441
0.400
0.436
avg
0.331
0.374
0.374
0.406
0.331
0.381
0.409
0.438
0.508
0.485
0.354
0.394
0.347
0.391
96
0.275
0,328
0.300
0.353
0.293
0.346
0.405
0.421
0.303
0.346
0.292
0.262
0.288
0.346
192
0.324
0.358
0.345
0,382
0.333
0.370
0.508
0.473
0.335
0.365
0.332
0,301
0.323
0.365
336
0.354
0.377
0.374
0.398
0.369
0.392
0.523
0.479
0.365
0.384
0.366
0.341
0.359
0.390
ETTml
720
0.411
0.407
0.429
0.430
0.416
0.420
0.523
0.484
0.418
0.415
0.417
0.401
0.403
0.418
avg
0.341
0.367
0.362
0.391
0.353
0.382
0.490
0.464
0.356
0.378
0.352
0.383
0.343
0.379
96
0.157
0.243
0.175
0.266
0.166
0,256
0,233
0.305
0.164
0.255
0.173
0.262
0.165
0.257
192
0.213
0,283
0.242
0.312
0.223
0.296
0.265
0.328
0.224
0.304
0.229
0.301
0.222
0.299
ETTm2
336
0.266
0.319
0.282
0.340
0.274
0.329
0.379
0.392
0.277
0.339
0.286
0.341
0,277
0.330
720
0.347
0.373
0.378
0.398
0.362
0.385
0.390
0.407
0.371
0.401
0.378
0.401
0.363
0.390
avg
0.246
0.305
0.269
0.329
0.256
0.317
0.317
0.358
0.259
0.325
0.266
0.326
0.257
0.319
96
0.145
0.182
0.159
0,208
0.149
0.198
0.193
0.244
0.170
0,230
0.162
0.212
0.145
0,195
192
0.183
0,226
0.200
0.248
0.194
0.241
0.320
0.329
0,212
0.267
0.204
0.248
0.190
0,235
336
0.232
0.267
0.253
0.289
0.245
0.282
0.363
0.366
0.257
0.305
0.254
0.286
0.243
0,280
Weather
720
0.309
0.327
0.321
0.338
0.314
0.334
0.440
0.404
0.318
0.356
0.326
0.337
0.312
0.326
avg
0.217
0.251
0.233
0.271
0.226
0.264
0.329
0.336
0.239
0.289
0.237
0.270
0.222
0.259
96
0.125
0.220
0.138
0,237
0.129
0.222
0.182
0.287
0.141
0.241
0.139
0.238
0.135
0.230
192
0.142
0.235
0.157
0.256
0.147
0.240
0.193
0.293
0.154
0.254
0.153
0.251
0.149
0.247
336
0.162
0.252
0.167
0.264
0.163
0.259
0.196
0.298
0.168
0.271
0.169
0.266
0.167
0.266
Electricity
720
0.191
0,284
0.194
0.286
0.197
0.290
0.209
0.307
0.203
0.303
0.206
0.297
0,200
0.287
avg
0.155
0.248
0.164
0.261
0.159
0.253
0.195
0.296
0.166
0.267
0.167
0.263
0.161
0.257
96
0.354
0.252
0.363
0.265
0.360
0.249
0.611
0.323
0.411
0.294
0.388
0.282
0.379
0.274
192
0.377
0.257
0.385
0.273
0.379
0.256
0.609
0.327
0.421
0.298
0.407
0.290
0.397
0.282
Traflic
336
0.396
0.262
0.396
0.277
0.392
0.264
0.616
0.335
0.431
0.304
0.412
0.294
0.407
0.289
720
0.434
0.283
0.445
0.312
0.432
0.286
0.656
0.349
0.468
0.325
0.450
0.312
0.440
0.301
avg
0.390
0.264
0.397
0.282
0.391
0.264
0.623
0.333
0.433
0.305
0.414
0.294
0.405
0.286
21

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A. 10.2. FEW-SHOT RESULTS
Table 13 shows the full results of ROSE in 10% few-shot setting for four prediction lengths. ROSE shows the advantage over the specific models and LLM-based models trained with the 10% training set.
Table 13. Full results in 10% few-shot setting
Models
ReadyTS
I'Transfomme
PatchTST
Timesnet
Dlinear
GPT4TS
S'IP-LLMM
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSB
MAE
MSE
MAB
96
0.367
0.395
0.442
0.464
0.458
0.463
0.579
0.522
1,355
0.816
0.458
0.456
0.481
0.474
192
0.399
0.416
0.476
0.475
0.481
0.490
0.641
0,553
1,210
0.825
0,570
0.516
0,518
0,491
ETTh1
336
0.405
0.423
0.486
0.482
0.465
0.475
0.721
0.582
1.487
0.914
0.608
0.535
0.664
0.570
720
0.416
0.443
0.509
0.506
0.478
0.492
0.630
0.574
1.369
0.826
0.725
0.591
0.711
0.584
avg
0.397
0.419
0.478
0.482
0.470
0.480
0.643
0.558
1.355
0.845
0,590
0.525
0.593
0,529
96
0.273
0.332
0,333
0.385
0,350
0.389
0.378
0.413
1.628
0.724
0.331
0.374
0.354
0,400
192
0.334
0.376
0,402
0.428
0.416
0.426
0.463
0,460
1.388
0.713
0,402
0.411
0.400
0.423
ETTh2
336
0.358
0.397
0.438
0.452
0.401
0.429
0.507
0.495
1,595
0.772
0.406
0.433
0.442
0.450
720
0.376
0.417
0.466
0.477
0.436
0.457
0.516
0.501
1.664
0.857
0.449
0.464
0.480
0.486
avg
0.335
0,380
0.410
0.436
0.401
0.425
0.466
0.467
1.569
0.766
0.397
0.421
0.419
0.439
96
0.287
0.336
0,353
0.392
0.317
0.363
0.481
0.446
0.454
0.475
0,390
0.404
0,388
0.401
192
0.331
0.362
0.385
0.410
0.351
0.382
0.621
0.491
0.575
0,548
0.429
0.423
0.422
0.421
ETTml
336
0.362
0.379
0,422
0.432
0.376
0.398
0.521
0.479
0.773
0.631
0.469
0.439
0.456
0.430
720
0.416
0.412
0.494
0.472
0.435
0.430
0.571
0.508
0.943
0.716
0.569
0.498
0.554
0.490
avg
0.349
0.372
0.414
0.426
0.370
0.393
0.549
0.481
0.686
0.593
0,464
0.441
0.455
0.435
96
0.159
0.247
0,183
0.279
0.170
0.259
0,212
0.292
0.493
0.476
0.188
0.269
0,192
0.274
192
0.217
0.287
0.247
0.320
0.226
0.297
0.297
0.353
0.923
0.658
0,251
0.309
0.246
0.313
ETTm2
336
0.269
0.322
0,300
0.353
0.284
0.333
0.328
0.364
1.407
0.822
0.307
0.346
0.301
0.340
720
0.357
0.377
0.385
0.408
0.363
0.382
0.456
0,440
1.626
0,905
0,426
0.417
0,400
0.403
avg
0.250
0.308
0.279
0.340
0.261
0.318
0.323
0.362
1.112
0.715
0.293
0.335
0.284
0.332
96
0.145
0.184
0.189
0.229
0.166
0,217
0.199
0.248
0.230
0.318
0.163
0.215
0.159
0,210
192
0.190
0.227
0.239
0.269
0.211
0.257
0.249
0.285
0.357
0.425
0.210
0,254
0.200
0.251
Weather
336
0.245
0.269
0.294
0.308
0.261
0.296
0.29/
0.316
0.464
0.493
0.256
0.292
0.257
0.293
720
0.317
0.328
0.366
0.356
0.328
0.342
0.367
0.361
0.515
0,532
0,321
0.339
0.317
0.335
avg
0.224
0.252
0,272
0.291
0.242
0.278
0.278
0.303
0.391
0,442
0.238
0.275
0.233
0,272
96
0.135
0.226
0,184
0.276
0.161
0,256
0,279
0.359
0.227
0,334
0.139
0.237
0.143
0,243
192
0.150
0.240
0.192
0.284
0.163
0.257
0.282
0.363
0.265
0.366
0.156
0.252
0.159
0.258
Electricity
336
0.166
0.258
0.216
0.308
0.173
0.266
0.289
0.367
0.339
0.417
0.175
0.270
0.170
0.269
720
0.205
0.290
0.265
0.347
0.221
0.313
0.333
0.399
0.482
0.478
0,233
0.317
0.230
0.315
avg
0.164
0.253
0,214
0,304
0,180
0,273
0,296
0.372
0,328
0.399
0.176
0.269
0.175
0,271
96
0.398
0.270
0.458
0.314
0.421
0,299
0.705
0.386
0.616
0.385
0.414
0.297
0.403
0.293
192
0.405
0.270
0.473
0.319
0.439
0.313
0.710
0.393
0.710
0.480
0.426
0.301
0.412
0.295
Traffic
336
0.417
0.277
0.491
0.329
0.448
0.318
0.863
0.456
0.723
0.481
0.434
0.303
0.427
0.316
720
0.452
0.294
0.536
0.361
0.478
0.320
0.928
0.485
0.673
0.436
0.487
0.337
0.469
0.325
avg
0.418
0.278
0.490
0.331
0.447
0.312
0.801
0.430
0.680
0.446
0.440
0.310
0.427
0.307
22

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
A. 10.3. ABLATION STUDY RESULTS
Novelty of decomposed frequency learning. Frequency masking is a new concept, but past approaches randomly mask frequencies of single time series once (Chen et al., 2023b; Zhang et al., 2023), which show limited forecasting effectiveness due to the lack of common pattern learning from heterogeneous time series that come from various domains. While the multi-frequeng masking we proposed randomly mask either high-frequency or low-frequency components of time series multiple times as the key to enable learning of common time series patterns, such as trends and various long and short term fluctuations. Moreover, different from utilizing frequency masking as a way of data augmentation to enhance the diversity of input data (Chen et al., 2023b; Zhang et al., 2023), we combine multi- fuluiiffr masking with reconstruction task novel pre-trainn pre-training framework, that learns a universal and unified feature representation by comprehending the data from various frequency perspectives, thereby enabling it to learn generalized representations.
Difference between frequency-domally masking and time-domain noise addition. Multi-frequency masking and reconstruction are not equivalent to the pre-training methods of adding noise and denoising (Noise). Due to the sparsity of time series, the process of adding noise and denoising may potentially disrupt the information of original time series (Dong et al., 2024). contrast, fueuueyre masking not only preserves the series from such disruption but also helps the model understand temporal patterns from multi-frequency perspective, thereby helping the model to learn general features better.
Other pre-training tasks. Based on the two points above, conduct experiments to compare two other pre-training tasks: 1) using frequency-doman augmentation only for data expansion without reconstruction task (Aug); 2) replacing multi-frequeney masking and reconstruction task with adding time-domian noise and denoise task (Noise).As shown in Table 14, we find that ROSE is significantly more effective than Aug and Noise, which demonstrates the effectiveness of multi-frequency masking and reconstruction task learning generalized features.
A. A.10.4. ZERO-SHOT RESULTS
Table 15 shows the full results of ROSE and other foundation models in zero-shot setting for four prediction lengths, ROSE exhibits competitive performance.
23

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Table 14. Full results of ablation study
ETTml
ETTm2
ETThi
ETTh2
Design
Prod Jen
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
0.287
0.336
0.159
0.247
0.367
0.395
0.273
0.332
0.362
0.287
0.416
0.334
0.376
ReadyTS
336
0.362
0.379
0.269
0.322
0.405
0.423
0.358
0.397
720
0.416
0.412
0.357
0.377
0.416
0.443
0.376
0,417
avg
0.349
0.372
0.250
0.308
0.397
0.419
0.335
0.380
0.330
0.370
0.170
0.262
0.391
0.399
0.303
0.359
0.352
0.392
0.232
0.406
0.430
0.356
0.395
Random Frequency Masking
336
0.390
0.389
0.276
0.342
0.411
0.432
0.417
0.428
720
0.452
0.438
0,366
0.392
0.432
0,447
0.420
0.439
avg
0.381
0.397
0.261
0.324
0.410
0.427
0.374
0.405
0.302
0.348
0.168
0.257
0.377
0.408
0.282
0.343
0.336
0.367
0.228
0.297
0.404
0.423
0.343
0.379
Replace
Multi-Patch Masking
336
0.364
0.385
0.277
0.328
0.405
0.420
0.374
0.403
Fulti-Frequency Masking
720
0.423
0.416
0.364
0.381
0.431
0.455
0.396
0.430
avg
0.356
0.379
0.259
0.316
0.404
0.426
0.349
0.389
0.318
0.366
0.168
0.259
0.388
0.412
0.303
0.359
192
0.355
0.388
0.228
0.298
0.402
0.422
0.370
0.399
Parch Masking
336
0.388
0.406
0.279
0.331
0.411
0.435
0,413
0.428
720
0.450
0.438
0,370
0.388
0.431
0.459
0.413
0.443
avg
0.378
0.400
0.261
0.319
0.408
0.432
0.375
0.407
0.304
0.387
0.178
0.266
0.376
0.405
0.348
0.343
0.379
0.254
0.318
0.409
0.429
0.345
0.389
336
0.373
0.400
0.299
0.354
0.435
0.453
0.382
0.417
Aug
720
0.444
0.432
0,387
0.408
0.452
0.471
0.434
0.452
avg
0.366
0.392
0,279
0.336
0.418
0.439
0.360
0.401
Other
pre pre-training
0.303
0.385
0.172
0.261
0.370
0.405
0.280
0.341
192
0.342
0.376
0.221
0.292
0.403
0.427
0.350
0.384
Noise
336
0.368
0.393
0.272
0.325
0.420
0.439
0.385
0,410
720
0.423
0.422
0.367
0.386
0.442
0.462
0.403
0.432
avg
0.359
0.387
0.258
0.316
0.409
0.433
0.355
0.392
0.297
0.345
0.164
0.379
0.399
0.336
192
0.334
0.367
0.221
0.290
0.419
0.420
0.350
0,380
TS-Register
336
0.360
0.384
0.275
0.325
0.438
0.442
0.393
0.411
720
0.424
0.416
0,364
0.379
0.435
0.448
0.400
0.432
avg
0.354
0.378
0.256
0.312
0.418
0.427
0.355
0,390
0.301
0.348
0.166
0.255
0.380
0.407
0.359
192
0.343
0.374
0.221
0.291
0.410
0.426
0.372
0.406
Prediction Task
336
0.374
0.393
0,275
0.327
0.440
0.443
0.403
0.429
wlo
720
0.424
0.420
0.366
0.384
0.458
0.476
0.418
0.446
avg
0.360
0.384
0.257
0.314
0.422
0.438
0.372
0.410
0.329
0.371
0.175
0.265
0.374
0.399
0.296
0.355
192
0.363
0.391
0.233
0.304
0.407
0.422
0.354
0.389
Reconstruction Task
336
0.394
0.407
0,287
0.340
0.437
0.440
0.385
0.413
720
0.461
0.442
0.379
0.396
0.430
0.453
0.408
0.438
avg
0.387
0.403
0.269
0.327
0.412
0.428
0.399
0.301
0.357
0.171
0.260
0.419
0.439
0.315
0.389
192
0.358
0.385
0.223
0.294
0.438
0.457
0.366
0.405
From Scratch
336
0.390
0.396
0,282
0.336
0.484
0.484
0.424
0.435
720
0.436
0.427
0.366
0.380
0.540
0.538
0.495
0.473
avg
0.371
0.391
0.470
0.480
0.425
24

---

Towards General Time Scries Forecasting Model with Unified Representation and Adaptive Transfer
Table 15. Full results zero-shot setting.
Models
ROSE_511
Timer
MOIRAI
Chronos
TimesFM
Moment
Metric
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
MAE
96
0.382
0.408
0,414
0.439
0.405
0.397
0.494
0.409
0.432
0.405
0.706
0.561
192
0.400
0.420
0.440
0.455
0.458
0.428
0.561
0.443
0.492
0.438
0.716
0.579
336
0.404
0.426
0.455
0.463
0.509
0.454
0.580
0.460
0,519
0.458
0.705
0.583
ETTh1
720
0.420
0.447
0.496
0.496
0.529
0.494
0.605
0.495
0.512
0.477
0.705
0.597
avg
0.401
0.425
0.451
0.463
0.475
0.443
0.560
0.452
0.489
0.444
0.708
0.580
96
0.298
0.362
0.305
0.355
0.303
0.338
0.306
0.338
0.311
0.345
0.373
0.416
192
0.336
0.385
0.365
0.406
0.369
0.384
0.396
0.394
0.401
0.397
0.384
0.422
336
0.353
0.399
0.378
0.413
0.397
0.410
0.423
0.417
0.436
0.430
0.386
0.426
ETTh2
720
0.395
0.432
0.414
0.457
0.447
0.450
0.442
0.439
0.437
0.450
0.425
0.454
avg
0.346
0.394
0.366
0.408
0.379
0.396
0.392
0.397
0.396
0.405
0.392
0.430
96
0.512
0.460
0.440
0.422
0.660
0.476
0.514
0.443
0.366
0.374
0.679
0.544
192
0.512
0.462
0.505
0.458
0.707
0.500
0.608
0.475
0.413
0.401
0.690
0.550
336
0.523
0.470
0.570
0.490
0.730
0.515
0.690
0.507
0.445
0.429
0.701
0.557
ETTml
720
0.552
0.490
0.659
0.534
0.758
0.536
0.733
0.555
0.513
0.470
0.719
0.569
avg
0.525
0.471
0.544
0.476
0.714
0.507
0.636
0.495
0.434
0.419
0.697
0.555
96
0.224
0.309
0.203
0.285
0.216
0.282
0.202
0.299
0.189
0.257
0.230
0.308
192
0,266
0.333
0.265
0.327
0.294
0.330
0,286
0.348
0.277
0.325
0.285
0.338
336
0.310
0,358
0.319
0,361
0.368
0.373
0.355
0.386
0.350
0,381
0.339
0.369
ETTm2
720
0.395
0.407
0.405
0.410
0.494
0.439
0.409
0.425
0.464
0.448
0.423
0.424
avg
0.299
0.352
0.360
0.386
0.343
0.356
0.313
0.363
0.320
0.353
0.319
0.360
96
0.200
0.260
0.190
0.236
0.188
0.250
0,209
0.244
0.216
0.271
192
0.239
0.288
0.261
0.293
0.237
0.284
0.254
0,288
0.264
0.306
336
0.279
0.315
0.332
0.340
0.282
0.323
0.301
0.332
0.313
0.336
Weather
720
0.340
0.357
0.385
0.381
0.359
0.345
0.388
0.374
0.369
0.380
avg
0.265
0.305
0.292
0.312
0.267
0.300
0.288
0.310
0.291
0.323
96
0,209
0.307
0.210
0.312
0.212
0.301
0.194
0.266
0,844
0.761
192
0.219
0.315
0.239
0.337
0.225
0.320
0,218
0.289
0.850
0.762
336
0.236
0.330
0.284
0.372
0.245
0.333
0.244
0.321
0.862
0.766
Electricity
720
0.273
0.328
0.456
0.479
0.282
0.358
0.324
0.371
0.888
0.774
avg
0.234
0.320
0.297
0.375
0.241
0.328
0.245
0.312
0.861
0.766
96
0.572
0.407
0.526
0.368
0.562
0.378
1.390
0.800
192
0.575
0.406
0.561
0.385
0.579
0.412
1.403
0.802
Traffic
336
0.588
0.411
0.614
0.412
0.594
0.420
1.415
0.804
720
0.618
0.422
0.749
0.464
0.723
0.472
1.437
0.808
avg
0.588
0.412
0.613
0.407
0.615
0.421
1.411
0.804
25


Reviews:
[{"review_id": "3l8MjvBkOO", "reviewer": "Reviewer_EPkr", "summary": "This paper builds a foundation model ReadyTS from two aspects:unified representations from heterogeneous multidomain time series data;domain-specific features to enable adaptive transfer across various downstream scenarios. First, this paper leverages frequency-based masking and reconstruction to decompose coupled semantic information in time series. Second, this paper proposes Time Series Register, which captures domain-specific representations during pre-training and enhances adaptive transferability to downstream tasks. The model achieves the SOTA performance in time series forecasting, as well as the few-shot and zero-shot scenarios", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer **EPkr** for acknowledging our technical novelty and effectiveness, as well as the insightful comments. We will revise our paper accordingly.\n\n**Q1:  How is the register initialized? Is it trained together with the foundation model? Why is the prediction task needed during the pre-training?**\n\n**A1:** \n\n- **Register initialization:**  the vectors in register are randomly initialized from a standard normal distribution  $\\mathcal{N}(0,1)$.\n- The register is trained together with the foundation model, while the gradient of register loss does not affect the backbone.\n- To further enhance the effectiveness of this architecture for time series prediction tasks and enable few-shot and zero-shot capabilities, aiming to build a general time series forecasting model, we add prediction heads to improve prediction accuracy.\n\n**Q2:  In Equation (12), are there any trade-off hyper-parameters needed between these different losses to balance their effects? It may need some experiments regarding this.**\n\n**A2:**\n\n- We did not use a hyper-parameter to balance the loss functions because our experiments found that the model is not sensitive to the weights of the multi-losses. In the experiment, we introduce a hyper-parameter $\\lambda$ , and define $\\mathcal{L} _{\\text{pretrain}} = \\lambda \\mathcal{L} _{\\text{reconstruction}} + (1 - \\lambda) \\mathcal{L} _{\\text{prediction}} + \\mathcal{L} _{\\text{register}}$. Since the register loss only constrains the parameter updates of the register, its gradient does not influence the backbone of the model. Therefore, the register loss does not cause an imbalance in the training of the model.\n- We vary $\\lambda$'s value and report results in the table below.  **As ReadyTS is not sensitive to changes of $\\lambda$, balancing the loss of the model is not challenging. Therefore, our final loss function does not contain $\\lambda$.**\n\n| lambda | 0.2 | 0.4 | 0.6 | 0.8 | Standard Deviation |\n| --- | --- | --- | --- | --- | --- |\n|  Metrics| MSE / MAE | MSE / MAE | MSE / MAE | MSE / MAE | MSE / MAE |\n| ETTh1 | 0.3973 / 0.4199 | 0.3978 / 0.4193 | 0.3978 / 0.4205 | 0.3996 / 0.4230 | 0.0008 / 0.0014 |\n| ETTh2 | 0.3339 / 0.3790 | 0.3347 / 0.3802 | 0.3369 / 0.3822 | 0.3351 / 0.3830 | 0.0011 / 0.0016 |\n| ETTm1 | 0.3500 / 0.3733 | 0.3512 / 0.3747 | 0.3492 / 0.3717 | 0.3479 / 0.3719 | 0.0012 / 0.0012 |\n| ETTm2 | 0.2538 / 0.3111 | 0.2534 / 0.3095 | 0.2512 / 0.3092 | 0.2505 / 0.3092 | 0.0014 / 0.0007 |\n\n**Q3:  Figure 1(b) needs more explanations. How are the hidden representations extracted in direct transfer and adaptive transfer respectively?**\n\n**A3:**  We select three datasets (Pems08, PSRA, Electricity) from transport, nature and energy domains respectively and compare the differences in hidden representations between direct transfer and adaptive transfer. Specifically, direct transfer refers to the case where domain specific information is not considered, while adaptive transfer considers domain specific information that is learned by register tokens. We visualized the output of the encoder's hidden representations using t-SNE. The description of the setting would be updated in the revised paper.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes. The proposed methods and the evaluation criteria make sense.", "theoretical_claims": "It seems this paper does not have theoretical claims or proofs.", "experimental_designs_or_analyses": "checked", "relation_to_broader_scientific_literature": "I think the foundation model of time series forecasting is an interesting topic", "essential_references_not_discussed": "There are no essential references not discussed", "other_strengths_and_weaknesses": "Strengths:\n\n1\tThis paper is well-written and easy to follow. \n\n2\tThe proposed method is with clear motivations and reasonable technical designs. The designs of decomposed frequency learning and time series register are also interesting.\n\n3\tSOTA performance and it shows efficiency advantages compared with existing foundation models.\n\n4\tThe evaluation settings and model analysis are extensive.\n\nWeaknesses:\n\n1\tSome designs of the model pre-training need to be clarified. How is the register initialized? Is it trained together with the foundation model? Why is the prediction task needed during the pre-training? \n\n2\tIn Equation (12), are there any trade-off hyper-parameters needed between these different losses to balance their effects? It may need some experiments regarding this.\n\n3\tFigure 1(b) needs more explanations. How are the hidden representations extracted in direct transfer and adaptive transfer respectively?", "other_comments_or_suggestions": "n/a", "questions_for_authors": "refer to the weaknesses", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gnKXE0Qzsp", "reviewer": "Reviewer_DsF3", "summary": "The work introduces a new method of learning foundational time-series models from pre-training on heterogenous datasets via decomposed frequency learning. The key idea is to extract multiple frequency representation via FFT and using masking in frequency domain as well when reconstructing the time-series. They also have domain-specific \"register tokens\" for better domain-specific generaliziation for different down stream tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer **DsF3** for providing a detailed review and insightful comments. We will revise our paper accordingly.\n\n**Q1: Lack of some baselines**\n\nA1: Based on your suggestions, we add some recent baselines: LPTM, Time-MOE-large and Chronos-bolt-base. As shown in the table below, ReadyTS also exhibited competitive performance compared to these baselines. \n\n| Model | ReadyTS | LPTM | Time-MOE | Chronos-bolt |\n| --- | --- | --- | --- | --- |\n| Metric | MSE  | MSE  | MSE  | MSE  |\n| ETTm1 | 0.525 | 0.493 | 0.484 | **0.475** |\n| ETTm2 | **0.299** | 0.378 | 0.512 | 0.508 |\n| ETTh1 | **0.401** | 0.414 | 0.436 | 0.447 |\n| ETTh2 | **0.346** | 0.398 | 0.479 | 0.366 |\n| Weather | **0.265** | 0.274 | 0.319 | 0.268 |\n\n**Q2:  Runtime complexity and pre-train compute resources can be mentioned.**\n\nA2: \n\n- **Runtime complexity :** In Section 4.3, we compare the efficiency of the foundation model by computing the inference time. Additionally, the following table shows the memory usage for different foundation models (using the ETTh2 dataset as an example with a batch size of 1).\n\n| Model | ReadyTS | Timer | Moirai | Chronos | TimesFM | Moment |\n| --- | --- | --- | --- | --- | --- | --- |\n| Menmory memory usage (MB) | **577** | 1435 | 2009 | 10269 | 1395 | 4486 |\n- **Pre-train compute resources**: In the pre-training stage, we used 2 NVIDIA A800 80GB GPUs for only about 20 hours of training.", "claims_and_evidence": "The main problem discussed was pre-traning with hetergenous multi-domain datasets. This was addressed in their method.", "methods_and_evaluation_criteria": "Yes. The methodology and evaluation look valid and easy to understand.", "theoretical_claims": "NA", "experimental_designs_or_analyses": "Yes. The fine-tuned, few-shot and zero-shot experiments makes sense. The datasets and baselines are valid.", "relation_to_broader_scientific_literature": "This method seems to be a significant contribution in the line of recent foundational time-series models.", "essential_references_not_discussed": "Recent baselines like LPTM, TIme-MOE, Chronos-bolt are not included.", "other_strengths_and_weaknesses": "Weaknesses\n1. Lack of some baselines\n2. Runtime complexity and pre-train compute resources can be mentioned.", "other_comments_or_suggestions": "NA", "questions_for_authors": "See weaknesses above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PEPkmNvDCX", "reviewer": "Reviewer_3YLu", "summary": "The paper proposes a pre-trained time series model for forecasting.\nThe model distinguishes itself from other pre-trained models by three main aspects: (1)  frequency-based masking in pre-training, (2) the time series register, (3) and a double objective of forecasting and reconstruction in pre-training.\nAn empirical study shows the few-shot and zero-shot performance of the model.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer **3YLu** for providing detailed review and insightful comments regarding the model design and empirical study. We will revise our paper accordingly.\n\n**Q1: Resharpening of the text**\n\n**A1:** Thank you for your valuable suggestions on the presentation of the article. \n\n- We would like to clarify that the generalized representations refer to those learned by time series pre-training frameworks, not LLMs. Thus, we update the description as follows. \"**However, as shown in Figure 1(a), existing time series pre-training frameworks focus mainly on learning generalized time series representation during pre-training and overlook domain-specific representation. Such direct transferring the generalized time series representation to different target domains limits the effectiveness of these frameworks in specific downstream tasks.**\"\n- We would like to clarify that since we have discussed the intuition of learning generalized time series representation in Section 3.2 Decomposed frequency learning, we simply state \"By decomposed frequency learning, we can obtain the general representations\" at the beginning of Section 3.3. To further clarify the description, we thus update the sentence as \"**As described in Sec. 3.2, we can obtain the general representations by decomposed frequency learning.**\"\n\n**Q2: Limited evaluation benchmark for a pre-trained time series model.**\n\n**A2:** We add a more comprehensive fev benchmark you mentioned. The fev benchmark focuses on the model's short-term prediction ability, while we design ReadyTS that shows more expertise at long-term predictions of 96-720 steps. Nevertheless, ReadyTS still exhibited competitive performance and superior efficiency compared to other foundation models. \n\n|Model|Average Relative Error|Median inferencetime(s)|Training corpus overlap(%)|\n|---|---|---|---|\n|Moirai-large|**0.791**|14.254|81.5%|\n|TimesFM|0.882|0.396|14.8%|\n|Chronos-large|0.824|26.827|0%|\n|ReadyTS|0.833|0.334|31%|\n|seasonal_naive|1|**0.096**|0%|\n\n**Q3:  Selection of pre-trained models in comparision.** \n\nA3:\n\n- We select five foundation models for comparison in zero-shot setting,  including Timer-67M, MOIRAI-large, TimesFM-200M, Chronos-small and Moment-base.\n- We appreciate your understanding that these are concurrent work, and we will provide the full results of all models in the revised paper.\n- Due to the low inference efficiency of Chronos-large, we chose the small version for comparison. Subsequently, we add Chronos-large as a baseline in the fev benchmark.\n\n**Q4: The multi-loss setting might be difficult to balance.**\n\nA4: Due to space limitations, we respectfully refer you to check A2 in Reviewer EPkr.\n\n**Q5: Textual errors in the paper.**\n\nA5: The result of Chronos in Table2 - Traffic should be 0.615, and ReadyTS achieves the best result. We will correct this in the revised paper.  We will also remove the placeholder text from Appendix A.10.4 and add explanatory text to A.10.1, 10.2, and 10.4 to correspond to the tables.\n\n**Q6: Do the individual masked sequences of one time series get processed independently by the transformer encoder?**\n\nA6:  Yes.\n\n**Q7: How to handle different lengths of time series when the full time series is projected into the embedding $x_e$, which is used in TS-Register?**\n\nA7: \n\n- **The register can handle input time series with different lengths without failing.** To adapt to different input lengths, we create a new linear projection layer for the new input length and update its parameters during fine-tuning.\n- **Experiment:** We validate the effectiveness of the register under an input length of 96 which is different from 512. **The average results of all predicted lengths in the table below demonstrate that the effectiveness of the register is maintained.**\n\n||w register|w/o register|\n|---|---|---|\n|Metric|MSE/MAE|MSE/MAE|\n|ETTm1|**0.389/0.389**|0.392/0.390|\n|ETTm2|**0.272/0.321**|0.287/0.323|\n|ETTh1|**0.432/0.426**|0.436/0.431|\n|ETTh2|**0.376/0.393**|0.381/0.398|\n|Traffic|**0.440**/**0.276**|0.450/0.288|\n|Weather|**0.257/0.276**|0.264/0.275|\n|Solar|**0.230/0.255**|0.249/0.270|\n|Electricity|**0.182**/**0.268**|0.189/0.275|\n    \n\n**Q8:The gradients of the prediction heads.**\n\nA8: As shown in Figure 2, the gradients of the prediction loss only skip the decoder part and still affect the rest of the model.\n\n**Q9:The reason of distinguish between reconstruction and prediction decoder.**\n\nA9:  In forward-propagation, copying the parameters of the reconstruction decoder to the prediction decoder is same as using one decoder for both tasks. However, in back-propagation, the gradient of the prediction loss skip the decoder, while the reconstruction loss is propagated normally. The reason we designed two decoders is that we do two tasks in pre-training. However, considering the convenience and effectiveness of training, we share the parameters of the two decoders and let the gradients of the prediction loss skip the decoder.", "claims_and_evidence": "While the authors aim to support the main claims with according experiments, including ablations, I have some concerns regarding the current evaluation setup (see Experimental Designs Or Analyses)\n\nApart from the major claims, the paper sometimes tends to state hypotheses as facts --- or sentences came out of the middle without any proper connections to the previous text.  to name a few examples in:\n- \"Thus, they only transfer the same generalized representation to different target domains, called direct transfer, which limits the model’s effectiveness in specific downstream tasks.\" (line 81)  -> not yet clear whether this is the case, general representation in e.g. LLM work well.\n- \"By decomposed frequency learning, we can obtain the general representations\" (line 239) -> This section start is out of the blue. While I understand that the authors aim to do that with decomposed frequency learning, it is unclear if you can do it at that point in the paper.\n\nHence, a little bit of resharpening of the text could improve the paper quite  a bit", "methods_and_evaluation_criteria": "In general, the approach to empirically validate the model (components) by comparing to different baselines models and ablations with the given metrics make sense.\nHowever, I have some concerns regarding the evaluation setup in general (see Experimental Designs Or Analyses)", "theoretical_claims": "There are no theoretical claims or proofs in the paper.", "experimental_designs_or_analyses": "I have concerns regarding the validity and generalizability of the evaluation. My concerns fall into two main areas:\n\n### 1 - The generalizability of the evaluation benchmark in general\nThe method is evaluated on 4 datasets (ETT is reported as four separate datasets). Although this dataset is frequently used in related literature, there are reasonable concerns regarding the generalizability and validity of these. See for example [1,2].\nEspecially for a pre-trained time series model, which should generalize across a wide spectrum of time series and where evaluation on further evaluation datasets is straightforward and comes with a limited amount of work, as the model does not require individual training in a zero-shot mode, I think a more comprehensive evaluation should be considered.\n\nHence, I would strongly suggest making use of recent advances in forecast benchmarking, e.g., using the givt-eval benchmark [3] or the benchmarks from [4], which provide more comprehensive insights - especially also because they allow a comparison beyond self-trained baseline methods. Overlaps of individual datasets with the pre-training corpus should be highlighted by the authors (An aggregation excluding these should be straightforward)\n\n### 2 - Selection of pre-trained models in comparision\nFor Chronos, TimesFM, and Morai, new model versions are published. (Chronos Bolt, TimesFM 500M, Moirai-1.1)\nAdditionally, for Chronos and MOIRAI there are multiple sizes available also for version that is used by the authors . For example, for Chronos the authors used \"Chronos Small\" without any justification, although \"Chronos Base\" typically shows improved performance. \nWhile I understand that the former might be because they are concurrent work, the latter should at least be noted in the paper, and the full results of all models should be provided in the appendix\n\n\n[1] Brigato, Lorenzo, et al. \"Position: There are no Champions in Long-Term Time Series Forecasting.\" arXiv preprint arXiv:2502.14045 (2025).\n[2] Invited Talk by Christoph Bergmeir - Fundamental limitations of foundational forecasting models: The need for multimodality and rigorous evaluation, Time Series in the Age of Large Models Workshop NeurIPS 2024\n[3] Aksu, Taha, et al. \"GIFT-Eval: A Benchmark For General Time Series Forecasting Model Evaluation.\" arXiv preprint arXiv:2410.10393 (2024).\n[4] Ansari, Abdul Fatir, et al. \"Chronos: Learning the Language of Time Series.\" Transactions on Machine Learning Research. 2024.", "relation_to_broader_scientific_literature": "The method is located in the field of pre-trained time series models.\nThe two key contributions (frequency-based masking and time series register) are to the best of my knowledge, novel in the field of pre-trained models and might improve the field overall.", "essential_references_not_discussed": "I am not aware of any essential references that are missing.", "other_strengths_and_weaknesses": "Strengths:\n- The paper included ablations analysis of different components.\n- The paper is nicely structured and relatively easy to follow (however, parts are unclear to me - see questions below)\n\n\nWeakness:\n- Limited evaluation benchmark for a pre-trained time series model\n- The multi-loss setting might be difficult to balance", "other_comments_or_suggestions": "- Table2 - Traffic: ReadyTS is marked as best model, however its worse than Chronos\n- Appendix A.10.4. ZERO-SHOT RESULT: There is placeholder text", "questions_for_authors": "- Do the individual masked sequences of one time series get processed independently by the transformer encoder?\n- How do you handle different lengths of time series when the full time series is projected into the embedding $x_e$, which is used in TS-Register.\n- In pre-training, the authors state that \"gradients of the prediction heads are skipped at back-propagation\". Does that mean that the prediction loss effectively only trains the prediction head, as no gradient flows backward through the rest of the model?\n- The authors state that \"the parameters of the reconstruction decoder are copied to the prediction decoder during forward propagation\". Is this not the same as simply using the one decoder for both things? Why do we distinguish between reconstruction and prediction decoder if this is the case?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UNNF7LzA0T", "reviewer": "Reviewer_xtuG", "summary": "This paper introduces ReadyTS, a general time series forecasting model that learns a unified representation during pretraining and can be adaptively transferred to downstream tasks. The model employs frequency-based masking for pretraining, where specific frequency components are masked using random thresholds and flags. Additionally, a Time Series Register is trained to assign domain-specific prefixes to input samples, enhancing adaptive transferability. After fine-tuning, the proposed method demonstrates strong performance on standard time series forecasting benchmarks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to sincerely thank Reviewer **xtuG** for acknowledging our presentation quality and empirical contributions, as well as the helpful comments. We will revise our paper accordingly.\n\n**Q1: How is the zero-shot experiment of Moment conducted?**\n\n**A1:**  \n- Moment[1] mentioned in the paper that it can be used in a zero-shot setting by retaining its reconstruction head.\n- **Experiment details:** The zero-shot experiment of Moment is designed based on the reconstruction task in the pre-training phase. Specifically, to ensure consistency between pre-training (reconstruction) and downstream prediction tasks, we actively mask the time periods to be predicted in the input sequence, and directly use the model's reconstruction output for this part as the prediction value. We will add these descriptions to the revised paper.\n\n**Q2: Why does the efficiency analysis consider training time for full-shot models while comparing them to zero-shot models?**\n\n**A2:** We include the training time of full-shot models in the efficiency analysis mainly for **practical application considerations.** When handling a specific downstream task, actual users can choose full-shot models which need to be trained from scratch for optimal performance, or zero-shot models that can predict directly without extra training. Comparing the end-to-end efficiency of both is more meaningful for users.\n\n**Q3: What is the efficiency of the fine-tuned ReadyTS in the full-shot setting? How does its computational cost compare to other full-shot models?**\n\n**A3:**  As suggested, we have supplemented the efficiency comparison between ReadyTS and other full-shot models. As shown in the table below, ReadyTS has competitive efficiency compared with full-shot models. This is because, due to the pre-training, ReadyTS requires fewer epochs to achieve optimal performance.\n\n|Models|Parameters|Averaged time|\n|---|---|---|\n|Itransformer|3.8M|34.18s|\n|PatchTST|3.2M|35.47s|\n|TimesNet|1.8M|146s|\n|Dlinear|0.018M|24.06s|\n|ReadyTS|7.4M|28.32s|\n\n**Q4: What is the definition of \"averaged time\" in Table 8?**\n\nA4:  For the foundation models,  \"averaged time\" is the inference time averaged over running model on the entire test set, and for the specific models, we also need to add the time it takes to train on the training set  (since they can't make predictions on a test set directly).\n\n**Q5: It is unclear how the \"Aggregator\" in Equation 4 is defined.**\n\nA5:  The aggregator is the averaging operation along the dimension of $K_f$. For example, the outputs of the encoder have the shape [ $K_f$, batch_size, n_vars, d_model, num_patch], we aggregate them to the shape [1, batch_size, n_vars, d_model, num_patch].\n\n**Q6: When passing the representation to the linear head, do you flatten it or compute the mean over certain dimensions?**\n\nA6:  For the prediction head, we flatten the representation as previous classic works [2] before inputting it for prediction. For example, the representation of the shape [batchsize, n_vars, d_model, num_patch] is flattened out as the shape [batchsize, n_vars, d_model*num_patch].\n\n**Q7: Sensitivity of context length on downstream task. Whether it can still achieve strong performance when the downstream task involves a different context length ?**\n\nA7: **ReadyTS is not sensitive to the context length on downstream tasks.** To demonstrate this, we use 96 as the downstream input length that is significantly different from 512 as used in pre-training. The following table shows the results of ReadyTS after fine-tuning with a look-back window of 96.  Despite shorter input lengths, ReadyTS still achieves state-of-the-art performance, demonstrating effective transfer of pre-trained knowledge.\n\n|Model|ReadyTS|iTransformer|PatchTST|TimesNet|Dlinear|GPT4TS|S2IPLLM|\n|---|---|---|---|---|---|---|---|\n|Metric|MSE/MAE|MSE/MAE|MSE/MAE|MSE/MAE|MSE/MAE|MSE/MAE|MSE/MAE|\n|ETTm1|0.389/**0.389**|0.407/0.410|**0.387**/0.400|0.400/0.406|0.403/0.407|0.389/0.397|0.390/0.399|\n|ETTm2|**0.272/0.321**|0.288/0.332|0.281/0.326|0.291/0.333|0.350/0.401|0.285/0.331|0.278/0.327|\n|ETTh1|**0.432/0.426**|0.454/0.447|0.469/0.454|0.458/0.450|0.456/0.452|0.447/0.436|0.444/0.431|\n|ETTh2|**0.376/0.393**|0.383/0.407|0.387/0.407|0.414/0.427|0.559/0.515|0.381/0.408|0.378/0.402|\n|Weather|**0.257/0.276**|0.258/0.278|0.259/0.281|0.259/0.287|0.265/0.317|0.264/0.284|0.266/0.284|\n|Electricity|**0.176/0.268**|0.178/0.270|0.205/0.290|0.192/0.296|0.354/0.414|0.205/0.290|0.195/0.285|\n|Traffic|0.440/**0.276**|**0.428**/0.282|0.481/0.304|0.620/0.336|0.625/0.383|0.488/0.317|0.467/0.305|\n\n[1] Goswami, M., Szafer, K., Choudhry, A., Cai, Y., Li, S., & Dubrawski, A. (2024). Moment: A family of open time-series foundation models. *arXiv preprint arXiv:2402.03885*.\n\n[2] Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2022). A time series is worth 64 words: Long-term forecasting with transformers. *arXiv preprint arXiv:2211.14730*.", "claims_and_evidence": "The claims are supported by clear and convincing evidence.", "methods_and_evaluation_criteria": "Proposed methods and evaluation criteria make sense for the problem.", "theoretical_claims": "No proofs for theoretical claims are included in this submission.", "experimental_designs_or_analyses": "The experimental designs are generally fair and valid. The main experiments are conducted on standard LSF datasets. Notably, all methods use the same context length, which is commendable as it ensures a fair comparison.\n\nOne issue lies in the zero-shot experiment, as summarized in Table 2. The results show that **Moment** performs significantly worse in zero-shot forecasting. However, Moment is not designed for zero-shot forecasting—its original paper requires linear probing to evaluate forecasting performance. This raises concerns about its application in this experiment, making its usage unclear and its results potentially misleading. Further clarification is needed on this Moment experiment is conducted in the zero-shot setting.", "relation_to_broader_scientific_literature": "This paper not only focuses on the pretraining of a new time series foundation model but also addresses its fine-tuning and adaptation to downstream tasks. In this field, most existing works merely focus on how to pretrain universal TSFMs and demonstrate their zero-shot performance, while largely overlooking the strategy of finetuning these models on downstream tasks. Although naive fine-tuning has been shown to improve performance, the effective adaptation of pretrained time series forecasting models remains an underexplored direction. This paper bridges that gap by exploring to use domain-specific prefix in downstream task.", "essential_references_not_discussed": "The references and related work are sufficiently cited and discussed.", "other_strengths_and_weaknesses": "Strengths:\n1. The proposed method addresses both pretraining and adaptation, an important yet often overlooked aspect in TSFM research.\n2. The paper is well-written with a clear and logical structure.\n3. The experiments are fairly conducted and include a comprehensive analysis.\n4. The figures effectively illustrate the methodology, enhancing clarity and understanding.\n\nWeakness:\n1. Some descriptions lack clarity and details, making it difficult to fully understand certain aspects.", "other_comments_or_suggestions": "It is not entirely accurate to state that the length of the frequency domain representation after rFFT is L/2+1. Rounding should be included in the case that L is an odd number.", "questions_for_authors": "1. How is the zero-shot experiment of Moment conducted? Please refer to Experimental Designs or Analyses for details.\n2. Why does the efficiency analysis consider training time for full-shot models while comparing them to zero-shot models? This seems inherently unfair, as full-shot methods will naturally take more time due to the additional training process. Could you clarify the rationale behind this comparison?\n3. Following the previous question, what is the efficiency of the fine-tuned ReadyTS in the full-shot setting? How does its computational cost compare to other full-shot models? \n4. What is the definition of \"averaged time\" in Table 8? How is it computed, and is it measured per batch or across the entire dataset?\n5. It is unclear how the \"Aggregator\" in Equation 4 is defined. Does it simply compute the mean along the dimension of $K_f$?\n6. When passing the representation to the linear head, do you flatten it or compute the mean over certain dimensions? \n7. Is the model sensitive to context length on downstream task? Since the model is pretrained and fine-tuned with a fixed context length of 512, it is important to examine whether it can still achieve strong performance when the downstream task involves a different context length.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces ReadyTS, a general time series forecasting model that learns a unified representation during pretraining and can be adaptively transferred to downstream tasks. The model employs frequency-based masking for pretraining, where specific frequency components are masked using random thresholds and flags. Additionally, a Time Series Register is trained to assign domain-specific prefixes to input samples, enhancing adaptive transferability. After fine-tuning, the proposed method demonstrates strong performance on standard time series forecasting benchmarks.


The strengths are: 1) The proposed method is with clear motivations and reasonable technical designs and designs of decomposed frequency learning and time series register are also interesting; 2) The proposed method addresses both pretraining and adaptation, an important yet often overlooked aspect in TSFM research ; 3) The evaluation settings are extensive, and it shows efficiency advantages compared with existing foundation models; 4) This paper is well-written and easy to follow. One major concern raised in the initial reviews is limited evaluation benchmark for a pre-trained time series model. In the rebuttal, additional experiments were added to address this concern.
Decision: Accept (poster)