CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting
Authors: Jianrong Ding
Zhanyu Liu
Guanjie Zheng
Haiming Jin
Linghe Kong
Venue: NeurIPS 2024 poster (2024)
Keywords: Dataset condensation
Time series forecasting
Primary area: deep_learning_architectures

Abstract: \textit{Dataset condensation} is a newborn technique that generates a small dataset that can be used in training deep neural networks (DNNs) to lower storage and training costs. The objective of dataset condensation is to ensure that the model trained with the synthetic dataset can perform comparably to the model trained with full datasets. However, existing methods predominantly concentrate on classification tasks, posing challenges in their adaptation to time series forecasting (TS-forecasting). This challenge arises from disparities in the evaluation of synthetic data. In classification, the synthetic data is considered well-distilled if the model trained with the full dataset and the model trained with the synthetic dataset yield identical labels for the same input, regardless of variations in output logits distribution. Conversely, in TS-forecasting, the effectiveness of synthetic data distillation is determined by the distance between predictions of the two models. The synthetic data is deemed well-distilled only when all data points within the predictions are similar. Consequently, TS-forecasting has a more rigorous evaluation methodology compared to classification. To mitigate this gap, we theoretically analyze the optimization objective of dataset condensation for TS-forecasting and propose a new one-line plugin of dataset condensation for TS-forecasting designated as Dataset \textbf{Cond}ensation for \textbf{T}ime \textbf{S}eries \textbf{F}orecasting (CondTSF) based on our analysis. Plugging CondTSF into previous dataset condensation methods facilitates a reduction in the distance between the predictions of the model trained with the full dataset and the model trained with the synthetic dataset, thereby enhancing performance. We conduct extensive experiments on eight commonly used time series datasets. CondTSF consistently improves the performance of all previous dataset condensation methods across all datasets, particularly at low condensing ratios.

CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting
Jianrong Dingl Zhanyu Liu!", Guanjje Zheng"f, Haiming Jin', Linghe Kong'
School of Electronic Information and Electrical Engincering, Shanghai Jiao Tong University
2 Zhiyuan College, Shanghai Jiao Tong University
(rafaelding, htt.:aa...0.iiigggee jinhaiming, linghe koogg edu.
Abstract
Datasel condensation a newborn technique that generates small dataset that can be used in training deep neural networks (DNNs) to lower storage and training costs. The objective of dataset condensation is to ensure that model trained with the synthetic dataset perform comparably to the model trained with full datasets. However, existing methods predominantly concentrate on classification tasks, pos- ing challenges in their adaptation time series forecasting (TS (Te-orreca))ing) This challenge arises from disparities in the evaluation of synthetic data. In elassifica- tion, the synthetic data is considered well-distilled if the model trained with the full dataset and the model trained with the synthetic dataset yield identical labels for the same input, regardless of variations output logits distribution. Conversely, TS-forecasting, the effectiveness of synthetic data distillation is determined by the distance between predictions of the models. The synthetic data deemed well- distilled only when all data points within the predictions are similar. Consequently, SSreorccg has a more rigorous evaluation methodology compared to classifi- cation. To mitigate this gap, we theoretically analyze the optimization objective dataset condensation for "S-forecasting and propose new one-line plugin of dataset condensation for TS TS--orecasting designated as Dataset Condensation for Time Series Forecasting (CondTSF) based on our analysis. Plugging CondTSF into previous dataset condensation methods facilitates reduction in the distance between the predictions the model trained with the full dataset and the model trained with the synthetic dataset, thereby enhancing performance. We conduct extensive experiments on eight commonly used time series datasets. CondTSF con- sistently improves the performance of all previous dataset condensation methods across all datasets, particularly at low condensing ratios.
Introduction
Dataset condensation is a strategy for mitigating the computational demands of training large models on extensive datasets. It is pointed out by previous works 32 that building foundation models [14]
10
44 on time series forecasting (TS-forecasting) have become a hot topic. However, fine-tuning these large models using full time series datasets can entail considerable computational overhead. Hence, the employment of dataset condensation techniques becomes imperative. recent years, various methods have been proposed in the field of dataset condensation, such matching- 52 36) and kernel methods [33] (55). To date, dataset condensation methods have achieved success in classification tasks, including image classification
35
based methods 49. 3 20 38
5
50
22 graph classification
4 25 27 and time series classification (26).
Co-primary author.
Corresponding author.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Madel tniend with gytthetc daines M.
FBB
Toxt Buto
Predicted label No errrror
Tod Crewe
Similar prediction
Model todeed witk fell dataett My
Gord nyntheti
data
300
83
120
16
30
Confessation
(4) Evaluntion of syathetic data for image clussification
Kir
Flits FRit' CeddT
Model tratedd wita gustketic dataur M,
Paintwise distance
Tost Eater
Large
Disómilar
Model trained (il chisset
prediction
Jyy
123 Ud0
289
Contresation Epecks
Condimatton Ejects
(b) Evaluation of synthotic data for time series forecasting
(c) Comparison of performance different methods
Figure I: Left: Difference between evaluation of dataset condensation for classification tasks and time series forecasting tasks. Right: Comparison in performance previous methods with and without CondTSF.
However, directly applying these dataset condensation methods designed for classification to the domain of time series forecasting (TS-forecasting) results performance degradation. The objective of dataset condensation is to generate a synthetic dataset so that when the model MM, trained with the synthetic dataset and the model M trained with the full dataset are given identical input, the two models output similar predictions. However, the concept of similar prediction differs between classification and foreaaac..,,... In classification, as shown Fig.Ta), predictions are considered similar if M, and M assign the same class label, irrespective of differences in the distribution of output logits. Conversely, in TS-foreeasting. as illustrated in Fig,T(b), the similarity of predictions from M, and M, is indicated by the mean squared distance of the predictions. The predictions are deemed similar only when all data points within the predictions are similar. This distinction in evaluation indicates TS-forccasting imposes more stringent criteria in discerning similar predictions compared to classification. poses a challenge that previous dataset condensation methods based on classification fail to provide adequate assurance for the similarity between predictions of M, and My within the realm of f--rorcastiigg
To mitigate the gap, we propose novel one-line dataset condensation plugin designed specifically for TS-forecasting called Condensation for Time Series Forecasting (CondTSFF) based on our theoretical analysis. We first formulate the optimization objective of dataset condensation for TS- forecasting. Then we transform the original optimization objective into minimizing the distance between predictions of M, and M, Furthermore, to minimize the distance between predictions of M, and M,, , we decompose the task into minimizing two terms, namely gradient term and value term. We theoretically prove that plugging CondTSF into previous methods minimize the value term and gradient term synchronously. Therefore, CondTSF serves as an effective plugin to boost the performance of dataset condensation for TS f'S-orrcas., As depicted in Fig [(),, plugging CondTSF into previous methods yields a significant enhancement in performance.
In short, our contributions can be summarized as follows.
To the best of our knowledge, we the first explore dataset condensation for forrrrcass,, We conduct a theoretical analysis of the optimization objective of dataset condensation for TS-forecasting, breaking it down into two optimizable terms facilitate improved optimization.
Leveraging insights from our theoretical analysis TS-orrccastn,, we propose a simple yet effective dataset condensation plugin CondTSF. Plugging CondTSF into existing methods enables synchronous optimization of the two terms, leading to performance enhancement.
We conduct extensive experiments on eight widely used time series datasets to prove the effectiveness of CondTSF. CondTTSF notably improves the performance of all previous dataset condensation methods across all datasets, particularly under low condensing ratios.
2
Related Works
Time Series Forecasting: Time series forecasting (TS-forecating) is the task of using historical, time-stamped data to predict future values. Previous works utilize different methods to achieve better

---

performance. These models can be mainly categorized into 3 types. (1) Transformer-basee Models: Transformer 40 have shown great success in natural language processing, and models based on 54 emerged in TSSffrrcasting fields. (2) MI.P-asee Models: Efforts to
transformers
53
42
24
use ML.P-based models have been put into TS-eaascag in recent years [47] since DLiinar[45]
triumph transformer-based models with simple MLP structure. (3) Patch-based Models: These
29 focused on learning representation cross patches instead of learning attention at each time point. Therefore they used patching strategy before feeding the data to transfomers.
models 34
48 28
Dataset condensation is task that aims at distilling a large dataset into a smaller one so that when a model is trained on the small synthetic dataset and the full dataset separately, the testing performances of the trained models are similar. Previous works related to dataset condensation can be divided into 3 classes below. (1) Coreset Selecting Methods: These methods aim at selecting data with representative features from source dataset to construct synthetic Methods: These methods aim at minimizing a specific metric surrogate model learned from source dataset and synthetic dataset. The defined metrics are different, including gradient [51] 18 46 features from the same class [1]. distribution of synthetic
Dataset Condensation:
dataset 1 12 37 39 (2) Matching-
data 50 52 and training trajectories 5 7 8. (3) Kerne--based Methods: These methods aim at obtaining closed-form solution for the optimization problem utilizing kernel ridge-tegression (20 33] In this way, the bi-level optimization problem of dataset condensation is reduced to a single-level optimization problem. Based on these results, the following works have made significant progress in different areas, including decreasing training cost and time time(55), improving performance 30 31].
Preliminaries
Dataset Condensation for TS- I'S-orecasting Given time series dataset, we split the dataset into train set and a test set. In this paper, we denote the train set as f and the test set as æ. We denote the synthetic dataset s. The synthetic dataset is a small dataset distilled from the full train set f. Train set f. test set æ, and synthetic dataset s are all vectors. We employ MM as a neural network parameterized by 6. Without losing generality,
suppose the model MM is using historical sequence Ze with length m to predict future sequence Tt mi+mmt+mt with length n. Given the test set æ, we formulate the test error of Mo as the error between the prediction of Mo test input Tec+m and the test label Lt+mit+ m-nn as shown in [
Lrest (Ma, x) A Σ (M mmtmmttmmmlll
During dataset condensation process, a distribution of initial model parameters Pa is available for training model parameter sampling, and the full train set f is available for condensation. Subsequently, a synthetic dataset is distilled from the full train set ff using dataset condensation methods. During testing process, initial testing model parameter 6 is sampled from Ps. Since 0o, is sampled in the testing process, it's unavailable during the previous dataset condensation process. Then model parameters sett and f.estt are obtained by training initial testing parameter to, on synthetic dataset and the full train set f respectively. The objective of dataset condensation is to ensure model Ma,. and Mar have comparable performance on test set æ. Therefore the practical optimization objective is to ensure that model Ma trained with synthetic dataset 8 minimizes the test error Liest on test set æc. The optimization objective is formulated as Eq
min Ltest
4 Method
Since test set & is not available during the dataset condensation process, the original optimization objective for dataset condensation in is is non-optimizable. To mitigate this gap, in the following sections, we transform the non-optimizable objectíve into two distinct optimizable terms. Then we develop methods optimize the two terms, thereby indirectly optimizing the original objective.
4.1 Decomposition
In this section, we decompose the optimization objective of dataset condensation in 2 into two optimizable terms for better optimization. In the testing process, the initial testing model parameter
3

---

Break down to optimizable term
Non-optimizable objective
Break down optimizable term
min Cien (M,,
Initial synthetic data
Distilled synthetic data
Full train
Parameter Matching
CondUTy
Random Sumple
min |00 0,
min Ciabel
8
Gradient Term Optimization
Value Term Optimization
Figure 2: Complete process of dataset condensation using CondTSF.
0o.test sampled from a distribution of initial model parameters Pg. Then we train Oo,estt on the synthetic dataset to get model parameter 0g s,ttest and train t0, the full train set f to get model parameter f ftest.. Given test dataset æ, the optimization objective is formulated as Eq
minLtest(Mep (Ma,. x)
where Lrest (Mo 2) Σ
MeMes (#e:)+m)) #e4ma++4mll?
Meanwhile, there a non-optimizable error e between the prediction model Ma and the true label from the test dataset, which is formulated in Eq
-t+mmttm-++ MM (Fea+m) +
(4)
Then we decompose the upper bound of Lrest (Mo x) into two terms, as shown in Thm We utilize Taylor Expansion in the proof of Thm For each real test data Tet+m we can arbitrarily
choose position t' and get synthetic data s Then we can perform Taylor Expansion with sp to obtain the value of MM (Featm) and Ma (Ze.tmm).
Given arbitrary synthetic data + the upper bound of the optimization objective of dataset condensation Cteat(M8,, (M,,, x) can be formulated as such
Theorem 1.
frest(Mo..ten 3) S Σ ||||| |IMo (Sua)4m) Moo (8trs +
Value Term
(5)
MB V (B (xittm
8t 4m))(+m)ll
Gradient Term
To prove Thm I we use linear models for further analysis since linear models can be both effective and efficient "S-forecasting(45) Given a linear model MM(w)) Oz,, its second and higher order gradient is zero. Therefore first-order Taylor Expansion is sufficient to obtain the accurate prediction of the model. Meanwhile, if Ma is non-linear model, we ignore the higher-order terms of Taylor Expansion. We prove T applying the property of the first-order Taylor Expansion and triangular inequality of norm functions. The complete proof App Hence we decompose the optimization objective of dataset condensation for TS--orecas into two optimizable terms, namely value term and gradient term. For value term, ensures Mo and Ma cess are similar in prediction values. For gradient term, it ensures the predictions of MM and Ma are similar in gradient. Optimizing these two terms can optimize the upper bound of the original optimization objective, and therefore indirectly optimize the original optimization objective in Eq 3
4.2 Gradient Term Optimization
We develop method to optimize gradient term in this section. Given linear model Ma(x) dx. its gradient on input is V.Mp(x) (a) T It indicates that the gradient of linear model on input is the parameter of the model. We apply Cauchy-Schward Inequality to the gradient term and get its upper bound. We reformulate the gradient term and get its upper bound shown in Eq 6
||( [I.. (8p1))44)) V.M (8p))+4)) (Frattm 8y m) 11 (Gradient Term)
[1(s,teest - f.,test) (Frrtm sp:44m)l 1l
(6)
5 110 - 0fteest 2 8p:044ml ||

---

Input Syathetic Data 8t'c'--m
8y m
Train Expert Model Prediction May (dea--))
Expert Model Prediction Ma (9esim)
Similar
May (se24m)
:
Mo (
Mg (Berrm))
Visualization of Original Data
Expat Syuthele Exper Madel Pedletiin
Empirically illustrate: Majacen (Bese (m) Ma Mo)((888 Moj
Moj(8en) (Bea mm)
M Ma((Ses (
Figure 3: Given the same synthetic data as input, all expert models trained on the full train set f provide similar predictions. The initial parameters of the models are sampled from the same distribution Po. The visualization of this figure utilized MDS algorithm for dimension reduction.
Since test data Fi is not available during the dataset condensation process, the distance between synthetic data and test data |æ Sp:t//mm is not optimizable. Therefore only need to optimize the distance between parameters ||0s,test f 1| All previous dataset condensation methods based parameter matching can minimize this distance. Here utilize MTT(3] as example to clarify the optimization process. The optimization objective of trajectory matching is
min [10 0s,test
|00 fitest 0u.
However, since Ox,eest and feestt are trained from testing initial parameter fo, ~ Po. they are not
available during dataset condensation process. Therefore, practice, we sample 0 ....... 6 Po as initial parameters during dataset condensation process. The initial parameters are trained synthetic dataset s and full train set f respectively to get 0 0....., , 0 and 09. ....... 0 Then we substitute 0s fetst and fottest in Eq 7 with parameters sampled dataset condensation, making the optimization objective optimizable. The practical optimization objective is shown in Eq
10;
11
min
(8)
10
0g and 09, 0.....,0 0 are sampled, trained, and stored in a parameter buffer before dataset condensation process. It can be concluded that using trajectory matching methods is intuitively minimizing the distance between 6; and fy for all initial parameters & Po. By minimizing the upper bound of the gradient term, trajectory matching methods indirectly optimize the gradient
In practice, 09....,07
4.3 Value Term Optimization
We develop an optimization objective to optimize the value term in this section. Since f is trained from fo,test it's unavailable in dataset condensation process. To mitigate this gap, we prove that although f.ttst is unavailable dataset condensation process, its prediction Mo (m)) still available. To prove this statement, we sample initial model parameters 69..... . o from Ps. are all trained with the same full train set f. After training, we get parameters ....... 0z. It observed that models Møg. .M, predict similarly given arbitrary synthetic data Sy' as input.
Then of ....... 8
Since initial testing parameter to is also sampled from the same distribution Pe and O is trained from fo using the same full train set f. the prediction of Ma test is similar to predictions of arbitrary expert model Møj. The conclusion is formulated in Eq
Most (Sy:e Rs Mag (Bva)mm) ss Moz (meartm) R RS Mør (Str))+)
5

---

Algorithm Dataset Condensation with CondTSF (MTT as backbone)
Input:
Synthetic data s; Parameter buffer {(0o, 0f));; Synthetic learning rate o;; Trajectory matching
epochs N; Total condensation epochs F;; Additive update ratio [3; Gap of epochs G between
using CondTSF
Output: Optimized synthetic data 8 Split into training sets {(8rt+mm $++m:++++a
2: for each condensation epoch in range E do
3: if e mod G * 0 then
4: Sample (fh fy)) from {(0u, 0,)}*
5: Initialize student parameter du t 8
6:
for each trajectory matching epoch j in range N do /MMTT(3] trajectory matching" // Train Ma with synthetic data
7:
8:
0j+1 f 0 a (VV (s)::mm), mm-m for all synthetic data Cparam + 1 / 1112
9:
10:
Update synthetic data s with respect to Cporam
/Optimize gradient term
else
12:
for each train sample (Sta+mm 81mma4++n ) in training sets do
13:
Choose an arbitrary expert model with parameter 8y from {(0,, 8,))
14:
+ Mag (8tt-m)
S++m:t+m+n
(1- 81+m:t-m+n + 3 y M*Opimizz value term*/
16:
return
Experiments have proved Eq 9 in Fig 3 As shown in Fig 3 for each synthetic data input 8paa4 (orange points), the predictions of corresponding expert models (yellow and blue points) are sim- ilar. Therefore, although f.test is unavailable in the dataset condensation process, its prediction M@ (sp:(++m) can be obtained using the prediction an arbitrary expert model Mgg (m) Now we reformulate the value term and transform it into a practical optimization objective. Firstly, We formulate the upper bound of the value term shown in Thm
Theorem 2. The upper bound of the value term can be formulated as such
Mo (mew)mm) -May, (Svaa4m) 11 < 2- Σ [| (Sva'-m) 88 Sy (10)
We prove Thm 2 by utilizing the triangular inequality and the prediction optimality of fstestt on synthetic data s. The complete proof is in App[A.2] Accodring Thm we obtain an optimizable upper bound of the value term. Therefore the optimization objective for the value term can be naturally defined as minimizing the upper bound of the value term, as shown in Eq IT
min Llabel where Ltabel = Σ [I (Str:4m) Sy +m+nl
(11)
According to Thm 2 label error Lladel is the upper bound of the value term. Therefore, by minimizing the upper bound of the value term, the value term is indirectly minimized.
4.4 CondTSF
In this section, we develop a one-line plugin called CondTSF to minimize the label error Liahel in [ so that the value term can be optimized. CondTSF is lightweight one-line plugin, no backpropagation or gradient is required during the update. CondTSF utilizes simple yet effective additive method to iteratively update the synthetic data 8 and minimize the label error Liahnt- In TS-forec,,,,ng, when generating training data, the data is usually sampled overlap from the dataset. Inspired by the overlap property, utilize additive method in CondTSF to gradually update the synthetic data to avoid vibrations. In the énh update iteration, CondTSF uses the prediction of expert model M (. (m) to update synthetic label 4y m-nn The update process is shown in Eq
(6+1) 414mat4444
B) By 4m+n +3 MM (5
(12)

---

Table I: Distill performance of different dataset condensation methods. For each method, Xmeans CondTSF not used, v /eanss CondTSF used, and + means the decreased percentage test error after CondTSF applied. Five synthetic datasets are distilled and the average and standard deviation are reported.
ConFTSF
RachangoRate
Weather
Electricily
Trafic
MAF
MSF:
MAF
MSE
MAR
MSF
MAR
MSE
Random
0.783-0090
1,070+00744
0.530+0.004
0.64740.199
0.840400117
1,02-0017
0884--00x
1.300+0.44
0.716-00000
0
0.530-00057
0.0++D00117
1.017-0046
1.296+0,021
DC
1,602=0.115
0.692+0.215
0 0.449-00055
0,467+0.044
0.794+0.014
0.987-0035
0.818-0012
1.265+00022
15.8%
27.8%
6.9%
11.7%
1.7%
2.9%
0.7%
2.4%
0.778=000084
0.96440. 136
0 0.509+0.005
0.538:00.085
0.767=0.012
0.840-0.119
0.742=0.010
1052=00024
0.195=00.007
0.001400.04
0.336-10.009
0,28410.007
0.391
0.284=0.004
0.494
0.579.0.037
75.0%
93.7%
36.0%
47.2%
47.6%
66.1%
33.4%
45.0%
D.683=00128
(1 0.806++0:244
0.474+0.049
0,492+0.067
0.755+D.011
08200000188
0.741-0.013
1057+00035
PP
0.191=0.006
0.058+0.003
0 0.304-000006
0.283=00.005
0.390+0.006
0.285-0.006
0.490=0.013
0.570=0.022
72.0%
92.8%
31.7%
42.5%
33.8%
45.
0.730:20.126
0.
0.522:0.0011
0.557.00.000
0.719
0.790
0.741
1063:10.51
TESLA
0.188.00:014
0.059110.008
0 0.29510.010
0.276:10.013
0.389
0.293_0.006
0.57610.016
0.730.0.0025
74.3%
92.7%
43.6%
50.3%
46.0%
62.9%
22.2%
31.3%
0.690.0.1133
0.81810.278
0.51110.037
0.535100004
081100019
0.705
105410,014
FTD
0.184:000005
0.0555+0.003
0.320+0.000
0.280::00044
0.396:00003
0.290-0.012
0.501=00021
0.587:00032
73.35
93.35
37.3%
47.6%
47.1%
65.6%
32.7%
44. 44.3%
0.616_20.137
0.702110.243
0.51540.0035
0.551LL0.035
0,75210.016
0.850_...27
0.74010.013
1013400026
0.190:00.010
0.05K10.006
0.32010.0015
0.290.10011
0.381
0.276-00005
0.196100016
0.582.10.25
70.66%
91.8%
37.9%
47.6%
49.4%
67.6%
33.09%
44.2%
Full
0.110:00001
0.02340.000
0.19..10.001
0:131:00000
0.312:00002
0.223-0.002
0.406:00003
0.492:000004
CondTSF
MAE
KTTml
MSE
MAE
KTTm2
MSE
MAE
ETTM
MIS
MAE
ET
MSE
Random
0.728.00033
0.99310.082
0.695100.011
0.8891000000
0,75610.035
0.59_000883
0.71910.037
101310.089
0.672.10.020
0.85910:008
0.631
0.7081100033
0.706100:03
0.913_0.118
0.627
0.69110.158
DC
0.66]:00012
0.833100018
0.5 0.591
0.60310004
0.678100034
0.873-0.070
0.601
0.6311000000
1,8%
3.1%
6.45
14.9%
3.6%
6.4%
4.1%
9.15
0.653=00019
0.771-+0.040
0.685+00:22
0,754:0.051
0.60
0.45-00025
0.719:00055
0827:000016
MTT
D.401-00004
0.502+00.00
0.347+0.028
0.202+00.228
0,53240.014
0.580-0.029
0.329-0005
0.205--00022
24.8%
34.9%
49.3%
73.3%
23.3%
31.3%
54.2%
D.660=0.014
0.788+00002
0.615:+0.093
0.620+00110
0.694+0.008
0.81-00018
0.633=0.052
0.757=0.086
PP
0.489-00005
0.491:00.01
0. 0.336+0:024
0,190:00025
0.527
0,574-0029
0.336:0004
0.211:00005
26.05
37.7%
24.1%
50.1%
72.
1.641-0.009
0.751 +0.01
0
0,570-0.210
0.674+0.013
0.03-00000
0.616-n0005
0.30000.54
TESLA
0.542-0007
0.622+00.08
0. 0.292+00001
0.155+0000
0.535+0.0200
0.588-0.048
0.332=0.007
0.208-00.066
15.4%
17.25
49.4%
72.8%
21.0%
27.7%
46.2%
67.0%
0.063=00.009
0.790+0.020
0 0.563-10.147
0.571#0:22
0.69340.016
0.857=0.044
0.625
0.666:1.240
0.494100.007
0.502.10:010
0.347.100012
0.200.10.012
0.52910.014
0.570=0.030
0.335100009
0.210100008
25.5%
36.5%
38.4%
65.0%
23.7%
33.1%
46.5%
69.1%
0.612-10.031
0.768.10.050
0.611100017
0.679110.090
0.6891000066
0.070
0.6
0.60010.215
DATM
0.505
0.56910..04
0. 00.00-10.006
0,167100.005
0.53210.028
0.582-00.06
0.33020.004
0.20910:003
17.2%
25.9%
75.4%
22.7%
45,9%
Fall
0.432:00.001
0.473:10000L
0.2301100001
0.113:0.000
0.389
0.339_0.004
0.276:0.002
0.166:000002
where 0 < 3 <
is the update ratio of this additive update method. This additive update process lowers the label error Liabel of 8 in each iteration exponentially, which can be formulated as
c(et1)
Σ || | +mmttl+mmm Me (8 + )|1
(13)
9)² -)
Σ
(8) | 4 214mma4mmn Ma (S + (1 (1- (1)
(
Since the update ratio has value 0 <B < 1, CondTSF lowers the label error Liahet exponentially in each update iteration and solves the optimization problem for the value term. As a plugin module, CondTSF is used to update once for every G iteration of parameter matching methods. In this way, the gradient term and the value term can be optimized synchronously. The algorithm is shown in Alg I We also formulate the complete condensation process using CondTSF, as shown in Fig
Experiment
5.1 Experiment Settings
Dataset Settings: The efficacy of dataset condensation methods is assessed across eight time series datasets. For all datasets, the model is set to be using 24 steps of data to forecast 24 steps of data. We
7

---

Table 2: Information and condensation settings of time series datasets.
ETTH&ETTT
ExchumgeRate
Weather
Electricity
Traffic
Dataset length
57600
14400 3.39%
7588 6.18%
52696
26304
17544
Distill mito
0.91%
1.8%%
2.74%
Distilled lengch 48
48
48
48
48
48
Table 3: Generalization ability of different dataset condensation methods. For each datasct and each method, MLP, LSTM. CNN are trained with the synthetic data distilled from DLinear expert models. For each architecture. five test models are trained. the average and standard deviation of MAE, MSE are summarized. The result of CondTSF is using MTT as the backbone.
RuceageRate
CNN
MEE
ESTM
CNS
ADNE
MAAE
XISE
XSS
XXAE
KIEE
MYE
1,245=33167
1217:0.11
049C29015
0,55120020
0570290006
0.85560017
DC
071358005
0.38003.11
0.1 534
0.500-00038
0.58
3.519.0.00 072
0.509 014
0:5a=00022
0.44640011
0.400-0.01 017
0.17=90166
0.53400004
0
04776111044
0 0401136533
0.49910044
04434-00885
MTT
0 0421-80007
0.30--2:00 000
0431
0.31540000
0.40452007
0.00:.000 010
028660 D06
0:26200044
0.299400077
0.09-0 206
0329-0018
0.276a50014
TESLA
0.316.100008
0:323231300
0.375-200977
0:302.11044
0L299 2222
0.26.16559
0.200.100012
000
FTD
0425480001
0.355-3100 106
0453640.012 012
0.300-0011
0.45-520225
0.39 05
0.8660 000
0221120005
0.300-60025
0.55--000 017
03547-0015
0.300-0011
RUB8S
0.35 (0.06
0LAN2 BU992
8:101 18.22
0:243 9.009
0.229 10886
0.288 8/051
1:331 10.00
8:281 00807
(1.56 BUISE
Kleeiicky
Traffoo
XXXI
MAL
MBE
XISE
Rankom
0.700200255
0031=33009
0.500220016
0.70200012
2017200004 (34
024320.005
1050:00015
1.10050001
DC
0 0776400007
0512=3.000 016
0.7524..00 D04
0 0.50
0. 160000010
3.997.0.00 003
0.33000 211
1035-0631
0. 000000012
D-989:0.00 650
3.87-0.00 D09
KIP
0 0.00-00014
0.8
--3 109
0,001-0000
0.741-00366
0761-0066
0-95--000 035
0238-0000
005--004
0.16-00117
1038--000 RCS
0253-600 018
L.044-0023
MTT
0 0460660009
0.34:33.00 010
0.66740.013 013
0.3 0.376:8015
15
030155008
3.015:0 010
0555500 060
0 0207:0006
0.634:50908
87788-6.0 011
D4SS5a 007
0.817:60077
PP
0.85-80008
0.85-0000 000
0461 54 60.006
0.36 0.388-00005
0331-2015
D.88-- 001
061760 D06
0.75
0.751-0006
0.510=00008
01700-0000 010
D:00--0 004
07754-000
PTD
0 11518.800 11518.80000
03902222005
01414-20011
(1339220890
0635222055
01262226518
11366220016
10790 00993201100
DWTM
0 0.501-0011
0.415-3.022
0.90544.000 015
0. 0.45:00023
0.511:00005
3.31 007
0.583 006
0. 0.002-0015
0.590-00034
0.709-0 000
5.99850.00 005
0.72650012
8.336 10882
0,331
0.220
8:272
B:423 08.08
9.08
11419
8:554
Ellmi
ETim:
Kasdon
10004
0L95210101
BAT7 MLD07
03001 14041
0.754 LUSM
S/W0L
RIP
0 556--0005
0.697-5.00 m10
0.5 0.5544 006
0. 0000-b018 8
39911-000 015
0085-0000 211
0. 116--0006
0 0.200-00311
01189-0000 015
3347-0001 031
0.201-0/24
FRuho
0.50-800077
0215-2201
Bol1 14 006
0.798-00618
0620-00011
0.88-6 666
0447-0041
0. 0.475-00857
0.395-0 169
0528-6-05 07S
0.544-0 L42
10
0 198--0057
05 053744.00 061
0.5 0339-0083
038165001
3665:00000 606
0.1 0116:0003
0 0.39920001
D1135-0 000
5
0008-0001 015
81167-0010
TESTA
0 0:519-0/001
0.53--2003 065
051344 007
05116-0007
0370-00013
0600--0000 600
0222-0
0 3115--005
0.27--00007
01135-0 106
0395--00 041
0211-00043
DAZM
0 200180000
3155
0534220018
0616331046
01292001000 1000
0. 0.153210004
0.20020000
BL149:00 6477
33773380000
0.23220022)
0.42260844
6499:00013
0:200:9:002
8.107:0088
0.340:20000
0.101:00006
0.733
0.33808/114
MLP
CNN
ESTM
MAY
MAA
XIAK
Rasdon
1.775-100222
10000 20014
0172200002
0/799 10
0363250229
0025201.044
0.118:20013
2391220104 041
0635200000
0.554200819
04481-01
01346650083
0 0665-00044
8 6:300-000 1877
0695541000 069
0
075--bo1s
0850-00000
0 605
033044 11S
0 0-555-0000
n 0885545436
D-455-000 054
04777-0805
87736-0015
MTT
0.06
0413-4000
0.000-0011
R883-00007
5907
6:30--0000 005
0330--0004
01100-00 40 040
0.366220855
TESEA
0:00-00:05
0431 02 mm
0
659536 005
0651-2013 3
0413-00005
0491 -0 015
0 0:304.0.00 267
0 3181-0056
11 0.36540004
0130--0 001
2. 3.90--0-01
FTD
0:666-0/088
6755-0005 m1
0614-0
0716-0001
0725-0001
01187 -0 817
8 130 001
0330--0077
CaxTTFF
0.397-8001
6429-0000
0 0:38000005
0.473-88005
8:556318666
8.108:0884
0.387:50005
0.066:00000
8332-11666
0.311:8/05
set the length of the synthetic dataset as 48, as shown in Table 2 Each synthetic dataset can only generate one training pair. We conduct experiments with two larger distill ratios as shown in App B
We plug CondTSF into existing dataset condensation models based on parameter matching, including DC(51], MTT PP[21] TESLA(]. FTD and DATM[LI] to prove the effectiveness of CondTSF. We also conduct experiments on non-parameter-matchin based methods, including DM[50) (52 KIP 3 FRePo[55] to prove that optimizing value term only also helps boost the performance. The experiment setting and results are shown in App E We use DLinear [45] as the expert model to perform dataset condensation since DLinear is a linear model.
Model Settings:
The source dataset first divided into a train set and a test set. All synthetic data is initialized by randomly sampling data from the train set. After a synthetic dataset is finished distilling. it is used to train another five models. After the five models are trained, they are tested on the test set. Their average mean absolute error (MAE) and mean square error (MSE) are recorded. We repeat the process above five times and report the average and standard deviation. While testing the generalization ability of the dataset condensation methods, DLinear [45 is used as the expert model to perform dataset condensation. Meanwhile, MI.P, LSTM[3], and CNN are used as test models when testing the generalization ability of the dataset condensation methods.
Metric Settings:
8

---

My 6.
Ry Boll
Gradient Term
Label
Value Term
Sreat
1.0
NTT
MT
NTT
NTT Cwetttt
20
WTT Cueeter
NTT Cout
0.7
60
Labed
4.4
0.4
0.2
20
160
200
1200
160
200
50
Condereation Epuctos
Comfematioon Equclos
Comdencation Epuclo
Figure 4: Changing trajectory of Left: parameter error which refer gradient term, Middle: label error which refer to value term and Right: test error during dataset condensation process.
Implementation Details: As plugin module, we test CFndTSS with all previous methods. Each synthetic dataset is optimized using standard training process according to the chosen backbone model. CondTSF is set to update every 3 epochs and the additive update ratio / is set to be 0.01. All the experiments are carried out on NVIDIA RTX 3080Ti.
5.2 Results
Single Architecture Performance: The results are summarized in Table For each backbone method, the first line shows the performance of the backbone model, the second line shows the performance of a backbone model with CondTSF, and the third line shows the percentage of reduction MAE and MSE after CondTSF is applied. There's considerable reduction error for all backbone models. The results suggest that CondTSF is effective in optimizing the value term and enhancing the performance in dataset condensation for TS-forecasting. However, using CondTSF on DC[51] is not as effective other methods. The reason is that instead of directly matching parameters, DC matches the gradient of parameters on loss in each iteration. Indirectly matching gradient leads to accumulating errors in parameters, making DC unable to lower parameter error effectively as directly matching parameters. Therefore CondTSF is not effective enough when applied to DC[ST].
Cross Architecture Performance: We also conduct experiments to evaluate the cross- architercee performance of dataset condensation methods. The results are summarized in Table We test all models on all datasets with MLP, [S3M and CNN test models. All synthetic data is distilled using DLinear (55) model as experts. We use MTT as the backbone for CondTSF. We observe that CondTSF based MTT outperformed all other previous models.
5.3 Discussion
Test Performance and Errors: We conduct experiments on ExchangeRate dataset with MTT and MTT+CondTSF. As shown in 4ig.4 trajectory of parameter error |19,-8,1 !|? |1
label error Ltabel and test error ftest through the distillation process are presented. Regarding the parameter error corresponding the gradient term, both MTT and MTT+CondTSF converge quickly, suggesting that the incorporation of CondTSF doesn't impact parameter alignment. As for the label error corresponding the value term, since the initial synthetic data s randomly sampled from the train set f and the expert model is trained by the train set f. the label error of 8 is small at the beginning. However, the utilization of MTT results an elevation of label error, whereas employing CondTSF effectively mitigates this increase in label error. During the test, MTT+CondTSF notably outperforms MTT by concurrently optimizing both the value term and the gradient term.
Limitations
The limitation this work is that we use linear models in our analysis so that the gradient a model on input is the parameter of the model. Therefore, only linear models like DLinear 45) are solid enough to an expert model for dataset condensation. The analysis no longer holds when comes to more complicated models. However, experiments in App D and App show that CondTSF is

---

also effective with non non-parameter maatrummtte-maccingg methods and non-linear models, which merits further exploration.
Conclusion
In this study, we provide abundant proof that previous dataset condensation methods based on classification are not suitable for dataset condensation for TS-forecasting. We elucidate that these earlier methods, predominantly focused on classification tasks, only address a portion of the opti- mization objective pertinent to TS-forecasting. To address this issue, we propose plugin module called CondTSF that can collaborate with parameter matching based dataset condensation methods. CF optimizes the optimization objective that previous methods have neglected and boosts the performance of dataset condensation methods on TS-oorccas,ing. We conduct experiments on eight widely used time series datasets and prove the efffettiveness of proof and method. CondTSF consistently enhances the performance of all previous techniques across all datasets, substantiating its effectiveness in improving dataset condensation outcomes for TS- IS-forecasting applications.
Acknowledgements
This work was sponsored by National Natural Science Foundation of China under Grant No. 62102246, 62272301, and Provincial Key Research and Development Program of Zhejiang un- der Grant No. 2021C01034. Part of the work was done when the students were doing internships at Yunqi Academy of Engineering.
References
[1] Olivier Bachem, Mario Lucic, and Andreas Krause. Practical coreset constructions for machine learning. arXiv preprint arXiv: 2017.
[2] Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. Tempo: Prompt- generative pre-trained transformer for time series forecasting. arXiv preprint (rXiv:2310.04944 2023.
[3] George Cazenavette, Tongzhou Wang, Antonio Torralba, Alexei Efros, and Jun-Yan Zhu. Dataset distillation by matching training trajectories. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 4750-4759, 2022.
[4] Yutian Chen, Max Welling, and Alex Smola. Super-samples from kernel herding, arXiv preprint arXiv:/203.3447 1203.342, 2012.
[5] Justin Cui, Ruochen Wang, Si Si, and Cho-Jui Hsieh. Scaling up dataset distillation to imagenet- Ik with constant memory. International Conference on Machine Learning, pages 6565-6590 PMLR, 2023.
[6] Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. decoder-only foundation model for time-series forecasting. arXiv preprint arXiv:2310. 10688, 2023.
[7] Jiawei Du, Yidi Jiang, Vincent YF Tan, Joey Tianyi Zhou, and Haizhou Li. Minimizing accumulated trajectory error to improve dataset distillation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 3749-3758, 2023.
[8] Jiawei Du, Qin Shi, and Joey Tianyi Zhou. Sequential subset matching for dataset distillation. Advances Neural Information Processing Systems, 36, 2024.
[9] Qizhang Feng, Zhimeng Jiang, Ruiquan Li, Yicheng Wang, Na Zou, Jiang Bian, and Xia Hu. Fair graph distillation. In Proceedings of the Advances Neural Information Processing Systems (NeurIPS), 2023.
[10] Azul Garza and Max Mergenthaler-Canseeo. Timegpt-1. arXiv preprint arXiv:2310.0352 2023.
10

---

[11] Ziyao Guo, Kai Wang. George Cazenavette, Hui Li, Kaipeng Zhang, and Yang You. To- wards lossless dataset distillation via arXiv:2310.05773, 2023.
trajectory matching. arXiv preprint
[12] Sariel Har-Peled and Akash Kushal. Smaller coresets for k-median and k-means clustering. In Proceedings of the twenty-first annual symposium on Computational geomelry, pages 126-134, 2005.
[13] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9():1733-1780,, 1997.
[14] Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin- Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. Time-Ilm: Time series forecasting by reprogramming large language models. arXiv preprint 2rXiv:2310.01722 2023.
[15] Ming Jin, Qingsong Wen, Yuxuan Liang, Chaoli Zhang, Siqiao Xue, Xue Wang, James Zhang, Yi Wang, Haifeng Chen, Xiaoli Li, et al. Large models for time series and spatio-temporal data: survey and outlook. arXiv preprint 1rXiv:2310.10190 2023.
[16] Wei Jin, Xianfeng Tang, Haoming Jiang, Zheng Li, Danqing Zhang, Jiliang Tang, and Bin Ying. Condensing graphs via one-step gradient matching. In Proceedings of the ACM SIGKDD Conference Knowledge Discovery and Data Mining (KDD), 2022.
[17] Wei Jin, Lingxiao Zhao, Shichang Zhang, Yozen Liu, Jiliang Tang, and Neil Shah. Graph condensation for graph neural networks. In Proceedings of the International Conference Learning Representations (ICLR), 2022.
[18] Jang-Hyun Kim, Jinuk Kim, Seong Joon Oh, Sangdoo Yun, Hwanjun Song, Joonhyun Jeong, Jung-Wa Woo Ha, and Hyun Oh Song. Dataset condensation via efficient synthetic-data param- eterization. In International Conference on Machine Learning, pages 11102-11118. PMLR, 2022.
[19] Joseph Kruskal. Multidimennsional scaling by optimizing goodness of fit nonmetric hypothesis. Psychometrika, 29(1):1-27, 1964.
[20] Sachyung Lee, Sanghyuk Chun, Sangwon Jung, Sangdoo Yun, and Sungroh Yoon. Dataset condensation with contrastive signals. In International Conference on Machine Learning, pages 1:352-12364 PMLR, 2022.
[21] Guang Li, Ren Togo, Takahiro Ogawa, and Miki Haseyama. Dataset distillation using parameter pruning. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2023.
[22] Haoyang Liu, Tiancheng Xing, Luwei Li, Vibhu Dalal, Jingrui He, and Haohan Wang. Dataset distillation via the wasserstein metric. arXiv preprint arXiv:2311.18533 18553,, 2023.
[23] Mengyang Liu, Shanchuan Li, Xinshi Chen, and Song. Graph condensation via receptive field distribution matching. arXiv preprint 2xXiv:2206.1369) 2022.
[24] Shizhan Liu, Hang Yu, Cong Liao, Jianguo Li, Weiyao Lin, Alex Liu, and Schahram Dustdar. Pyraformer: cowpcomply pyramidal attention for long-range time series modeling and forecasting. In International conference on learning representations, 2021.
[25] Yilun Liu, Ruihong Qiu, and Zi Huang. Cat: Balanced continual graph learning with graph condensation. In Proceedings of the IEFE International Conference on Data Mining (ICDM), 2023.
[26) Zhanyu Liu, Ke Hao, Guanjie Zheng, and Yanwei Yu. Dataset condensation for time series classification via dual domain matching. arXiv preprint arXiv:2403.07225,, 2024.
[27] Zhanyu Liu, Chaolv Zeng, and Guanjie Zheng. Graph data condensation via self-expressive graph structure reconstruction. arXiv preprint arXiv:2403.07294, 2024.
11

---

[28] Zhanyu Liu, Guanjie Zheng, and Yanwei Yu. Cross-city few-shot traffic forecasting via traffic pattern bank. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, pages 1451-1460, 2023.
[29] Zhanyu Liu, Guanjie Zheng, and Yanwei Yu. Multi-scale traffic pattern bank for cross-city few-shot traffic forecasting. arXiv preprint arXiv:2402.0033/ 2024.
[30] Noel Loo, Ramin Hasani, Alexander Amini, and Daniela Rus. Efficient dataset distillation using random feature approximation. Advances in Neural Information Prucessing Systems, 35:13877-13891, 2022.
[31] Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation with convexi- fied implicit gradients. arXiv preprint arXiv:23302.06755, 2023.
[32] John A Miller, Mohammed Aldosari, Farah Saeed, Nasid Habib Barna, Subas Rana, Budak Arpinar, and Ninghao Liu. A survey of deep learning and foundation models for time series forecasting, arXiv preprint arXiv:2401.13912, 2024.
[33] Timothy Nguyen, Zhourong Chen, and Jaehoon Lee. Dataset meta-learning from kernel ridge-regression. In International Conference on Learning Representations, 2020.
[34] Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers, arXiv preprint arX1v:2211.14730, 2022.
[35] Kashif Rasul, Arjun Ashok, Andrew Robert Williams, Arian Khorasani, George Adamopoulos, Rishika Bhagwatkar, Marin Biloš, Hena Ghonia, Nadhir Vincent Hassen, Anderson Schneider, et al. Lag-llama: Towards foundation models for time series forecasting. arXiv preprint 10.0v22310.08278. 2023.
[36] Ahmad Sajedi, Samir Khaki, Ehsan Amjadian, Lucy Liu, Yuri A Lawryshyn, and Konstanti- nos N Plataniotis. Datadam: Efficient dataset distillation with attention matching. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17097-17107, 2023.
[37] Ozan Sener and Silvio Savarese. Active learning for convolutional neural networks: A core-set approach. arXiv preprint arXiv: 00489, 2017.
[38] Seungjae Shin, Hcesun Bac, Donghyeok Shin, Wconyoung Joo, and Il-Chul Moon. Loss- curvature matching for dataset selection and condensation. In International Conference Artificial Intelligence and Statistics, pages 8606-8628. PMLR, 2023.
[39] Ivor W Tsang, James T Kwok, Pak-Ming Cheung, and Nello Cristianini. Core vector machines: Fast svm training on very large data sets. Journal of Machine Learning Research, 6(4), 2005.
[40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[41] Kai Wang, Bo Zhao, Xiangyu Peng, Zheng Zhu, Shuo Yang, Shuo Wang, Guan Huang, Hakan Bilen, Xinchao Wang, and Yang You. Cafe: Learning to condense dataset by aligning features. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12196-12205, 2022.
[42] Haixu Wu, Jichui Xu, Jianmin Wang, and Mingsheng Long, Autoformer: Decomposition trans- formers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34:22419-2222 -24300, 2021.
[43] Zhe Xu, Yuzhong Chen, Menghai Pan, Huiyuann Chen, Mahashweta Das, Hao Yang, and Tong Hanghang. Kernel ridge regression-hasee hased graph dataset distillation. In Proceedings of the ACM SIGKDI) Conference on Knowledge Discovery and Data Mining (KDD), 2023.
[44] Hao Xue and Flora D Salim, Promptcast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions Knowledge and Data Engineering, 2023.
12

---

[45] Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? Proceedings of the AAAI conference on artificial intelligence, volume 37, pages 11121-11128, 2023.
[46] Lei Zhang, Jie Zhang, Bowen Lei, Subhabrata Mukherjee, Xiang Pan, Bo Zhao, Caiwen Ding, Yao Li, and Dongkuan Xu. Accelerating dataset distillation via model augmentation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 11950-1 11999, 2023.
[47] Tianping Zhang, Yizhuo Zhang, Wei Cao, Jiang Bian, Xiaohan Yi, Shun Zheng, and Jian Li. Less more: Fast multivariate time series forecasting with light sampling-oriented mlp structures. arXiv preprini arXiv:2207.01 186, 2022.
[48] Yunhao Zhang and Junchi Yan. Crossformer: Transformer utilizing cross-dimension dependency for multivariate time series forecasting. In The eleventh international conference on learning representations. 2022.
[49] Bo Zhao and Hakan Bilen. Dataset condensation with differentiable siamese augmentation. In International Conference on Machine Learning, pages 12674-12685. PMLR, 2021.
[50] Bo Zhao and Hakan Bilen. Dataset condensation with distribution matching. In Proceedings of the IEEE/CVF Winter Conference Applications of Computer Vision, pages 6514-6523, 2023.
[51] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. ICLR, 1(2):3, 2021.
[52] Ganlong Zhao, Guanbin Li, Yipeng Qin, and Yizhou Yu. Improved distribution matching for dataset condensation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 7856-7865, 2023.
[53] Haoyi Zhou, Shanghang Zhang, Jieqi Peng. Shuai Zhang. Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference artificial intelligence, volume 35, pages 11106-11115, 2021.
[54] Tian Zhou, Ziqing Ma, Qingsong Wen, Xue Wang, Liang Sun, and Rong Jin. Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International conference on machine learning, pages 27268-27286. PMLR, 2022.
[55] Yongchao Zhou, Ehsan Nezhadarya, and Jimmy Ba. Dataset distillation using neural feature regression, Advances in Neural Information Processing Systems, 35:9813-9888 2022.
13

---

A Complete Proof
A.1 Complete Proof for Theorem 1
Theorem Given arbitrary synthetic data +m- the upper bound of the optimization objective of dataset condensation Ltest (Mas, x) can be formulated such
Creat (Mo .) ||ll + ||Mes (sv:)+m) Ma (
Value Term
(14)
[( (Bes)4m))
Ma (8 (Brst+m 4
Gradient Term
Proof. Replacing the true label #++mastmmmn in Leeat (M coot 2) with Eq 4 the optimization objective of dataset condensation for TS-forecasting is reformulated as the distance between the predictions of Me and May. given the same test input. Then the triangular inequality of norm functions used and the original optimization objective can be transformed to its upper bound,
as shown in Eq 15
Lrest MMeseeM F.)
Mo (f))tt))))))) Tt Dt+mit++ull
Σ MlMesseer (Ttetm)) Mayac (Tt:tm)) - -El1?
(15)
1 Σ II (FEt4)) - Ma (xes++))l + ||el?
In Eq 15 we prove that minimizing the distance between Ma, ,ewt (Ies+m) and Mo r.est (X is equivalent minimizing the upper bound of the original optimization objective. Then decompose the distance between predictions of Me...e. and Mer. into two optimizable terms for better optimization. We use linear models for further analysis since linear models can be both effective and efficient in TS-forecasting [45) Given a linear model Ma(w) = Ox, its second and higher order gradient is zero, i.e. (arr) = 0, vk 2 2. Therefore, first-order Taylor Expansion can be utilized get the prediction of the model Mo test data mmtt using the prediction and gradient of the model Mg on arbitrary synthetic data se.4+m- The process is formulated Tq Meanwhile, is worth mentioning that if .M is non-linear model, then the second and higher-order terms of the Taylor Expansion are ignored in I6
Mo(Dretm) = Ma (o( + V.Mo
(16)
Then we expand Eq [5 with Taylor expansion. After that, the triangular inequality of norm functions is used to get its upper bound. In the meantime, by applying the triangular inequality, the optimization objective can be decomposed into two optimizable terms.
(MM
Σ || (Xrttm) Ma (Featm) 11 2 + ||e||²
Σ ||ll + |I (sva)mm) + Mo (Sv:U+m) (x -m:344m)
Mo (svartm) V.Mo (Sv:)+m)) (wustm stt:0+m)11
Σ ||c|| + 1I.o (Sva)mm) Mo (St))4m)
+ +V.Me (Str)+m) V.M (Stratm)))) (Fartm 8p:04m))l
|||| + ||M (Svartm) - Ma (Str: +m)=+
Value Term
I. (8r:04m)
V MMa (Btr)+m)) (Beitm 8pa44mm ||
Gradient Term
(17)
Therefore Thm is proved.
14

---

A.2 Complete Proof for Theorem 2
Theorem
The upper bound of the value term can be formulated such
Mo (Bea)4m) - -Mo (Suar ml? S 2. Σ MM (Su 8t mmal (18)
Proof. We first use triangular inequality and the non non-tegativity of norm functions to get the upper bound of the value term. The process is shown in Eq [q
Masien (m)))t+mm)
Ma (m (Value Term)
[M (8)))+m)
8t +m+a + 88 mat' +mtn Moracc (Sest I11
Sy 4 +m+ml
s| [sy- - Ma (S
Σ ||Me (Str)+m) Sy mm
||s Me (S (Sp:U+mm)
(19)
For further analysis, we need to step back and formulate the training process of 0g to derive an inequality. By doing dataset condensation, synthetic dataset is obtained. Then formulate the training process of sest on synthetic data s as minimizing the prediction error on s. The training process is formulated in Eq 20
0 teat arg min Σ (M 30
(20)
Now we can derive an incquality. We denote s as the synthetic dataset obtained by dataset con- densation. We denote Moz.cer as the model that is trained on as shown 2q According to Eq 20 MMg...a has the lowest prediction eтrоr on synthetic data s under the given model architecture. Since Mg....r and MMg share the same model architecture, the prediction error of Me.,.... on synthetic data s is no larger than the prediction error of Mf,test on synthetic data s. This inequality can be formulated as such
[MMo (Bt Sy tm in 1| $
Σ sy +m:t +m Mor (s (21)
By applying Eq
21 to Eq 19 obtain the upper bound of the value term, shown in Eq 22
Mo (s) Mo (sM 11 (Value Term)
Mo (m) 8r4mt/
Σ
|| +mon Ma (Br:)++m)1
2
(22)
Therefore Thm 2 proved.
15

---

B Performance with Different Distill Ratio
We further explore the performance of CondTSF with different distill ratios and compare the results with previous matching-base -ased methods.
B.1 Standard Ratio Condensation
We distill the dataset into synthetic dataset with a flexible length for each dataset. The information on condensation in Table 4
The performance is shown in T
Table 4: Information and condensation settings of time series datasets.
ETTmi&ETT22
ETTh1&ETT22
ExchangeRate
Weather
Electricity
Traffic
Dataser length
S7600
14400
7588
S2696
26304
17544
0.2%
1%
Distilled length 115
57
75
105
78
70
Table 5: Distill performance of different dataset condensation methods. For each method, Xmeans CSF is not used, / /cans CSn is used, and + means the decreased percentage of test error after CondTSF is applied. Five synthetic datasets are distilled and the average and standard deviation are reported.
CondTSF
RachangeRate
Weatber
Electricity
Traffic
MAA
MSE
MAE
MSE
MAE
MSE
MAE
MSE
Random
0.730.00168
0.957.0.3977
0.566:10.056
0.708:10.135
0.83210.021
L.80_00058
0.845000007
1.34310.017
0.657-00025
0.729110.02
0.1881000066
0.523100007
0.79710.011
0.973-0.031
0.81610.077
1.27710.023
0.645=0014
0.710+0.039
0.450:0.041
0.469:000009
0.769:0041
0.920=0.104
0.100:0017
1.250=00039
1.9%
2.6%
7.7%
10.4%
3.6%
5.5%
0.7%
0.5%
X
0.167-00.018
0.36110.024
0.33010.020
0.2911000200
0.433000014
0.379-0.017
0.57510.022
0.726.0.016
MTT
0.180-000008
0.05311.001
0 0.205100.100
0,25310.005
0.35510.002
0.238-00001
0.02910.005
0.50010.007
61.6%
25.5%
31.2%
0.463-00002
0.352:10.02
0 0.34010.000 022
0.301:00.022
00
0.375-0.009
0,582:00012
0.71400.114
PP
D. 0.179-00006
0.05++0.004
0 0.279-0.005
0.248-00005
0.356+0.003
0.240-0.001
0.423-n005
0.400-0.007
61.2%%
84.9%
17.7%
17.6%
28.5%
35.9%
27.4%
31.3%
D.406=00026
0.275+0.088
0334+00.009
0.292400008
0.530+0.007
0.463-0.008
0.650=00018
0,85--0.044
TESLA
0.185=00014
0.056++0.088
0.30
0.273-0.002
0.511=00012
0.614+0.020
54.5%
79.5%
12.4%
10.2%
30.5%
41.1%
21.4%
28.2%
0.445=00.038
0.332:00050
0.32410.010
0,2842200144
0,57010.003
0.374_0.004
0.557
0.680:100088
FTD
0.173=00008
0.949+0.002
0 0.274-+0.006
0.246+00004
0.329+0.004
0,232-0.005
0.410=0.005
0.476-0.004
61.24%
85.15
13.4%
30.0%
26.4%
30.0%
0.454=0.000
0.345:0.047
0315440.02
0.279=00001
0.495
0.10000.006
0.583=0.017
0.722=0.033
DATM
0,102:0.003
0.054100002
0 0.296:0.001
0.26420.007
0.325
0.208=0.002
0.410:0.006
0.473:00000
59.9%
$4.4%
5.8%
5.5%
34.3%
44.3%
29.7%
34.5%
0.110:00.011
0.02310.000
0.19710.001
0.131100001
0.31240:002
0.223-0.002
0,406100003
0.49210.004
CondTSF
KTTml
ETTm2
ETThi
ET
MAE
MSE
Random
0.097:0.054
093440.105
0.62910.129
0.74740.285
0.725:0:007
0.995=0.122
0.645=0.118
0.763:0.251
0.665=0.012
0.837+0:024
0.575+0.015
0,574200030
0.713+0:024
0.933-0.020
0.591=0.009
0.619#0.132
DC
0.659=0.010
0.82810.021
0,54210.078
0.516.00.18
0.89510.019
0,901=0.039
0.488100092
0.42910.148
1.1%
10.0%
17.5%
30.8%
0.18610.016
0:47810.021
0.326100013
0.183100013
0.63910.020
0.008_0.000
0,56110.116
0.55
MTT
0.470.000033
0.470100005
0.27310.010
0.133:10.007
0.453
0.422-00.016
0.324
0.197.0.003
3.4%
1.6%
16.35 3%
27.2%
29.1%
43.6%
42.6%
64. +4.2%
0.492:20.014
0.451100021
0.327:1.0117
0.51710.193
PP
0.166L00.003
0.47010.005
0.26310.006
0.127.000011
0.45110.003
0.020
0.335L00002
0.20910.003
5.3%
3.1%
19.5%
31.0%
30.5%
45.0%
38.4%
59.5%
0.500
0.55510..002
0.31510.005
0.1721:0001
0.61110.00
0.748-00020
0,54810.106
0,51910.115
0.514:00010
0.554+0.021
0.289:00.005
0.152:0.003
0,507
0.524-00019
0.334:0009
0.209-000010
3.1%
0.3%
11.8%
20.9%
30.0%
39.1%
59.7%
0.400-00006
0.47++0.010
0 530++00117
0:186-00017
0663++0.011
0.730-0.021
0.611=nmmss
0.62--0.064
FTD
0.463=00.008
0.466++0.003
0 0.264+00007
0.128=00.05
0,427+0.003
0.379-0.006
0.313=0.004
O.
0.186+00.00
5.4%
2.1%
19.9%
31.3%
32.6%
48.0%
48.8%
70.1%
0.514:00002
0.520+00015
0.323 005
0.179:00004
0.623
0.722-0.054
0.537:00
0.501:2..75
DATM
D. 0.400-000007
0.407+0.009
0. 261 +0.007
0.141+00006
0.425+00004
0.372-0.005
0.303-00005
0.175-H1.09
3,2%
4,4%
13.0%
21.5%
32.0%
48.4%
43.6%
66.25
Full
0.432=00001
0.47540.001
0 230+000001
0.113#0.001
0.389+0.003
0.339-01.04
0.276=0.0022
0.166++0.002
16

---

B.2 3-times Standard Ratio Condensation
We distill the dataset into a synthetic dataset with a flexible length for each dataset. Each synthetic dataset is 3 times larger than the synthetic data in Table The information on condensation is shown in 6 The performance is shown in 7
Table 6: Information and condensation settings of time series datasets.
ETTm&&ETTm2
ETTI&ETTT
ExchungeRate
Weather
Electricity
Trallic
Datase length
S7600
14400
7588
S2696
26304
17514
Discill milo 0.6%
1.2%
3%
0.6%
0.9%
1.2%
Disiiled langth
227
Table 7: Distill performance of different dataset condensation methods. For each method. Xineans CSn is not used, /mans CondTSF used, and & means the decreased percentage of test error after CondTSF is applied. Five synthetic datasets are distilled and the average and standard deviation are reported.
ConFTSF
RachangeRate
Weather
Eleetricity
Trafte
MAE
MSE
MAE
MSE
MAE
MAE
MSE
Random
0882-00.81
1.255+0222
0.44740.007
0.47++0.105
0.82+000066
1079-0.441
0.840=00011
1330+00052
0.711-0008
0.864+0.003
0.439+0.027
0.444-0085
0.27700008
1068-00118
0.83-0006
1. 1.307-0.077
DC
1.614=0.117
0.65
0 0.396=00:013
0.372=00019
0.804+0.003
L012-0.009
0.816=0.055
+.27
13.6%
23.8%
9,8%
16.3%
2.8%
5.2%
2.0%
2.6%
0.201=00012
0.066100008
0.29310.027
0.242=00003
0.520:1.008
MIT
0.175510.006
0.05010.002
0.27410.003
0.25510.007
0.331
0.241_0.003
0.422.0.006
0.505
12.8%
23.7%
15.6%
12.7%
0.3%
0.1%
2.3%
2.9%
D. 0.198=0.008
0.06240.005
O 0.300:0..015
0.277=00015
0.33340.004
0.242-0.005
0.435=00005
0,522=00.00
0.176=00.033
0.05110.002
0
0.350
0.219=0.0022
0.429
0,512:1.005
11.0%
19.9%
11.0%
6.5%
1.0%
1.2%
1.4%
1.8%
0,20910.016
0.07110.011
0.29710.005
0.2651000033
0.51610.0011
0.31
0,59310.011
0.734.0.023
TESLA
0.176.00.009
0.051110.05
0 0.287.10.005
0.262100001
0.13100007
0.336_0.009
0.551
0.66110.011
27.9%
7.3%
7.1%
9.5%
00008
0.064:10:005
0 0.32810.015
0.298:1.017
0.353
0.243-0.004
0.435:00005
0.523-00.05
FTD
0.172-00004
0.049+00.002
0 261 +0.007
0,258-00004
0.351+0.000
0.243-0.004
0.421 0.421 -0000
0.501-0.005
13.3%
23.0%
14.3%
13.4%
0.7%
0.0%
3.3%
4.3%
0.196110.010
0.062.10.05
0.437100005
DATM
0.173:000007
0.049:100003
0.275:10.005
0.251:10.00)
0.326110003
0.208-0.003
0.416:00005
0.49:1000003
12.0%
21.3%
3.0%
4.8%
2.8%
2.2%
4.6%
5.0%
Full
0.110=00000
0.02++0.000
O. 097--00000
0:131+0000
0,312+0.002
0223-0000
0.406-D000
0.49--0.004
CondTSF
KTTml
KTTm2
ETTh
ETTh2
MAE
MSE
MAE
MSE
MAB
MSE
MAE
MSE
Random
0.603.00041
0913100005
0.629100065
0.72410.155
0.74210:055
L.27_00119
0.69110.140
0.857.0.291
0.6031000055
0.730100005
0.19010.018
0410010032
0.724100007
0.977-0.022
0.63410.054
0.71110.115
DC
D. 0.500=00009
0.713400.025
0.417
0,312:50:16
0.704:00002
0.915-00006
0,566:00008
0.562-10.11
2,2%
2.4%
14.8%
24.0%
2.8%
6.3%
10.7%
21.0%
D:52D=00022
0.522+00.05
0.143+00005
0A00-D0009
0.467-0011
0.329-0009
199-0..00
0.462=00.06
0,476+000022
0.265-+0.099
0.130=00007
0.42++00009
0.383-0.012
0.303-0.007
0.177=0.000
11.1%
8.8%
6.9%
9.1%
10.9%
18.0%
7.8%
11.0%
0.538=0.041
0.558#0.002
0 0.200-000088
0.144:00007
0.477#0.006
0.462-0.012
0.330=0.0044
0.201=0.004
0.466-00006
0,45+00.022
0 0.271 +0.099
0.135-00008
0.442+0.016
0.405-00029
0.323-n004
0.198-00.05
13.34
13.25
5.7%
7.3%
2.4%
0.519=00014
0.558+0.050
2.20500.005
0:155=00004
0.542+0.055
0.603-0.057
0.339=00006
0.213-+0.005
TESLA
0.480=00:255
0.507:+0.001
0 0680000004
0.152=0.004
0.480=0.010
0.471=0.020
0.327=00005
0.205=+0.004
7.5%
8.2%
2.5%
1.7%
11.5%
21.9%
3.3%
3.8%
0.501
0.53910:006
0.480:100009
0.464=0.018
0.327
0.197
FTD
0.169LL0.005
0.193110.088
0.261100003
0.130.00.002
0,41010.006
0.100_0.012
0.308
0.18110.009
11.7%
8.5%
9.75
13.3%
8.3%
13.7%
5.8%
7,8%
0.197
0.51310.001
0.205.00.006
0,144100005
0,8001000022
0.161_0.026
0.327
0.19610..005
0.403:000006
0.495:00009
0.268-H.0008
0.131:20.007
0.429-0.033
0.385-00053
0.299=0.007
0.172::0.007
0.9%
3,4%
5.7%
8.6%%
10.6%
17.1%
8.7%
12.1%
Full
0.402-10.001
0.473100001L
0.23010.001
0.113:10.00
0.389
0.339-0.004
0.2761000022
0.166.1.002
We observe that CondTSF consistently improves the performance of backbone models with all condensing ratios, suggesting the effectiveness of CSndTSF with different condensing ratios.
17

---

C Performance of CondTSF with Non-paramelle-mmatciig Based Methods
We distill the dataset using the standard condensing ratio. The information on condensation is shown in Table We conduct experiments on CondTSF with non- pat---aaaa-aaatceteeeee based methods. We use 50], 52M K3P FRePo[55] as backbone methods. The performance shown in T
Results show that using CondTSF optimize only one of the two optimizable terms also boost the performance.
Table 8: Distill performance of different dataset condensation methods. For each method, Xmeans CSndTSF is not used, means CondTSF is used, and & means the decreased percentage of test after CondTSF applied. Five synthetic datasets are distilled and the average and standard deviation reported.
CondTSF
ExchangeRate
Weather
Electricity
Traflic
MAE
MSE
MSE
Random
0.730=0.168
0.957100397
0.566:00.056
0.708 135
0.832
L.80=0.058
0.845
1344:20017
0.772=00016
0.990:1.0011
0.483-10.063
0.540:1..26
0.81810.011
L048-0034
0.830
1.299:0.048
0.832.10.022
0.51311.0022
0.81710.011
1.53110.035
9.6%
16.0%
1.2%
5.0%
0.1%
0.1%
2.3%
3.5%
0.708-00107
0.87110.257
0.517.1.0522
0,59110.105
0.8361000022
L067_00032
0.823
1.27710.055
IDM
0.683:0.120
005:10.22277
0. 0.504:00.055
0,570:10.19
0.81920.023
1050-0005
0.804200000
1.231
7.5%
4.4%
0.508-000026
0,467+00.02
0.316+H0.06
0.297+0.008
0.17+000000
1,40-00032
0.34=00006
1.314-00277
KIP
0.217.00.009
0.079100007
0.313.10.077
0,297110.033
0.8121000011
1.07_00.044
0.83010.011
1.278.0.031
59.698
83.2%
1.0%
0.0%
0.6%
0.3%
0.4%
288
0.518:00:00
0.40110.045
0.61520015
0.789.00037
FRePo
0.270=00021
0.122:40.01
0.330+0.031
0.28K-0.03)
0:464:00011
0.373-0.010
0.518=0011
0.601100221
47.8%
74.1%
22 22.19
28.4%
21.2%
32.6%
15.8%
23.8%
Pall
0.110=0.001
0,02++2.000
O. 09700000001
0.131+00001
0,312+0.002
0,223 0.012
0.406-n0005
0.492+0.004
CondTSF
ETTml
ETTm2
ETTbl
ETTh2
MAE
MSE
MAE
MIE
MAB
MSB
MAE
MSE
Random
0.607:00054
0.629:00.129
0,747:0
0.645:0.118
0.763-00.51
0.684200063
0.90300.129
0.641:00.1
0,782.10
0.722:10.040
0.977-0.094
0.703:000099
0.895:0.189
1MM
B661--0041
0.826++0.009
0.614+0.130
0.706++0
0.713+0.035
0.950-0.077
0.615-0.135
0.706-68266
4.8%
8.5%
4.3%
9.7%
1.2%
2.8%
12,5%
21.1%
0.657=0047
0.84140.00M
0.648-0.155
0.81+002997
0.71340.055
0.956-0.124
0.667=0.121
0,833-0.252
0.816+00:400
0.694
0.573
1.4%
3.0%
5.8%
13.9%
2.6%
4,7%
14.1%
23.2%
D.581=00002
0.736+H0.022
0316:+0.002
0.171:00002
0.685HD0211
0861-0002
0.576=0.114
0,575-0. 198
KIP
0:581=00001
0.725+00014
0.20++0.002
0.151+00002
0.60240.036
0.709-0.083
0.400-n.054
0.262-00061
1.8%
30.6%
51.0%
0:596=0.015
0.670#00004
0.572+0.023
0.556:20.05
0.640#0.004
0.759-0.024
0.549=0.0077
0.528:00.111
FRoPo
0.55 1.50011
0.58110.011
0.42410.024
0.303-10.039
0.566:0.005
0.617=0.008
0.430:0.064
0.325:00079
7.6%
13.2%
25.8%
45.5%
11.6%
18.7%
21.6%
38.4%
0.380
0.276110.002
18

---

D Performance of CondTSF with Non-Linear Expert Models
We distill the dataset using the standard condensing ratio. The information on condensation is shown We conduct experiments on distilling dataset with non-linear expert models. We use MTT TESLA and DI as backbone methods. The performance of using a CNN as the expert model is shown in Table and the performance of using 3-layer-MLP as the expert model shown in Table I0
in Table 4
Results show that CondTSF is also effective with non-linear expert models.
Table 9: Distill performance of different dataset condensation methods with CNN as the expert model. For each method, Xmeans CondTSF is not used, / /mans CondTSF is used, and means the decreased percentage of test error after CondTSF is applied. Five synthetic datasets distilled and the average and standard deviation reported.
ConFTSF
MAA
ExchangeRate
MSB
MAB
Weatber
MSE
MAB
Electricily
MSB
MAB
Traflic
MSB
Random
0830
1.02200127
0.50V110.01
0.526100000
0,762:000066
0.882-0034
0.732:00023
1.04::0.051
0.372=00.28
0.257+0:031
0.31440.013
0,482:00009
0.906:000266
MIT
0.140:00001
0.063100001
0.21610.008
0.120110.004
0.357.0.005
0.267-0.005
0.451
0.51910.009
62.4%
73.4%
21.7%
56.8%
25.9%
32.1%
31.9%
42.79
0.378_00.007
0.212110.007
0.31010.015
0.292110:012
0.516.00005
0:130_00.009
0.655
0.900
0:134:0012
0.05K10.002
0.253:10.007
0.137::0:002
0.374:00008
0.267=00007
0.528:0011
0.632:10,200
64.65%
76.0%
18.4%
53.1%
27.5%
37.9%
19.45
0.331=00011
0.179H0.013
0.335+0.009
0.294:0.008
0.504+00000
0.412-0.009
0.587=00000
0.74:-0.014
DATM
0.137-0017
0.059+0.004
0.291+0.000
0.261+00004
0.355+0.006
0.252-0.006
0.452-0.099
0.526++0.011
58.6%
67.0%
13.1%
11.2%
29.6%
41.7%
23.0%
29.1%
CondTSF
KTTml
kTTm2
ETTM
ETTB2
MAE
MSE
MAR
MSR
Random
10601-00012
0.856+0.000
0.723+00027
08846-0.022
0.727+0.015
0931-00000
0.722-0.015
0.847-0003
0.550-00006
0.565++0.D11
0.205+00000
0.644+0.113
0.788-0.88
0.371-0016
0.245-0.020
MTT
0:482-00077
0.507++0.007
0 236+000009
0.111=0005
0.460+0.008
0.47-0004
0.297=0.011
0.173+0.009
12.4%
13.3%
32.0%
45.9%
28.6%
44.5%
19.9%
29.4%
0.544:00000
0.5K3++0:007
0.35910.013
01222:00006
0.64:10.010
0.757-0.037
0.365200008
0.00000000
TESIA
D. 1.409-00.008
0,492+0.007
0 +025190.009
0.119-00005
0.473+00009
0.458-0.01
0.203-n004
0.170-0.002
9.3%
15.6%
30.15
46.4%
25 25.4%
39.5%
19.7%
29.25
0.566-000088
0.598+00.011
0.318++0.013
0.174#0.014
0.653+0.009
0.773-0.025
0.349-0.012
0.221--0012
0.518=0.007
0.521+00:012
0.23/=00002
0.107=0.001
0,451
0.290=000077
0.165-00.00
8.5%
12.9%
27.4%
38.5%
28.8%
45.9%
16.9%
25.3%
Table 10:
Distill performance of different dataset condensation methods with 3-layer-MLP as the expert model. For each method, Xmeans CondTSF is not used, / mcans CondTSF is used, and I means the decreased percentage test error after CondTSF is applied. Five synthetic datasets are distilled and the average and standard deviation are reported.
CondTSF
MAF
RxchangeRate
MSB
MAE
Weather
MSE
MAE
Electricity
MSE
MAE
Traflic
MSE
Random
0.932-00.08
124330.144
0.562:000013
0.6421000000
0.79610.013
0.955=0.036
0.75120.012
1.11240.025
0.364=00.02
0:11110:000
0.475
0.380=0.016
0.63310.011
MIT
0.139.000004
0.034110002
0.21810.015
0.135100.066
0.37510.007
0.268_0.008
0.501
0.58910.016
61.8%
83.9%
20.3%
54.5%
21.1%
29.5%
20.9%
30.0%
0.352:20.012
0.20910.010
0.29100.001
0,272110003
0.525100001
0.162-0.008
0.596100000
0.711100022
0:128:00117
0.027++0.07
0.252-H0.006
0.135::0.00
0.39740012
0.297-00014
0.48
0.577:40009
63.655
87.15
15.35 15.2%
50.4%
35.7%
17.8%
0.326-00016
0.177+0.016
0 549+0.0022
0.901+0000
0.517+0.00
0.441-0.011
0.622-N0007
0.785-H0005
DATM
0.141=00008
0.042+00001
0 2544+0.010
0.126=00004
0.385+00009
0.280-0.000
0.496=0.006
0.582+0.011
56.7%
20.3%
CondTSP
KTTml
kTTmm
ETTH
FT112
MAE
MSE
MAE
MIE
MAE
MSE
MAE
MSE
Random
1.022=00115
0.85500:227
0.762+0.012
0.955-0.057
0.687+0.010
0888-00.022
0.763=0.019
0945-00000
0.564-0.022
0.65++0.022
0.362+0.011
0.219-00012
0.61540.022
0.732-0.015
0.354=0.011
0.228--0012
MTT
0.403-00000
0.511100007
0.24:10.022
0.09810011
04423-00006
0.369-00008
0,28510000
0.345500.00
12.65
22.0%
32.
19.5%
0.501-00001
0,570+0.00M 004
0.341-40011
0.197:00033
0.006
0.745-0.022
0.337=0000S
0.210:08006
TESLA
0.487-00005
0.405+000004
0 0.250+0.009
0.119:00.005
0.58+-0.011
0.398-0.015
0.283-0.007
0.145-0.006
10.0%%
13.5%
26.7%
39.6%
27.7%
46.6%
16.0%
31.0%%
0.558=00.099
0.61140.021
0.329+00.006
0.186:00005
0,59340.0022
0.741=0.063
0.350=00009
0.222-00.008
0.498
0.48210.006
0.234:-0.005
0.107-:0.001
0.43740.006
0.397=0.005
0.286100005
0.167:10.004
10.8%
21.1%
28.9%
42.5%
26.3%
46.4%
18.3%
24.8%
19

---

E Performance Comparison of CondTSF and Smoothing
We distill the dataset using the one-shot condensing ratio. The information on condensation is shown in Table We conduct experiments on distilling dataset with CondTSF and a low pass filter respectively. We use DLinear [45] as the expert model. The result is shown Tab
Results indicate that using CondTSF is significantly more effective than using a low pass filter to smooth the distilled data.
Table 11:
Distill performance of different dataset condensation methods. Five synthetic datasets are distilled and the average and standard deviation are reported.
RxchangeRate MSE
Weather
Electricity
Traffic
MAE
MSE
Random
0.783:40.990
1.000:0.266
0.530=0.084
0.64700.199
0.840000177
1.02=00031
0.854=0.018
1.50040.043
MTT
0.77840.084
0.96410.136
0.509:0.065
0.538-0.85
0.747:0.122
0,800200199
0,742=00010
1.05240.024
1.35810.3211
0.60210.018
0.0042
1,208.0.UU
Mmoch
0.620--0.024
0.636++0.062
0.501 +0,012
0.527-0.028
0.788+00022
0.963+00061
0.83600.022
1.291-00.057
MTT+Cond CondTSP
0.195:0..007
0.061
0.009
0.284 0,007
0.091
0.284:0.000
0.494 0.022
0.579:10.037
ETTml
ETTm2
ETTbl
ET
MAE
MSE
MAE
MSE
MAE
MSE
MAE
MSE
Random
0.728:000033
0.993:0.082
0.69520.011
0.88900.330
0.756200.055
1.09920.083
0.749:0.037
1.013:10.099
MTT
0.653-#0.019
0.771=20.040
0.68==0.022
0.754=0.051
0.693=10.009
0.845:0.023
0.719=0.006
0.827+40.016
Smooth
0.728:10.013
0.991:10.038
0.706:00.053
0.100-01136
0.701
0.87910.022
0.744=0.067
1.023:0. 145
Smoohh
0.919+B.040
MTT-CondTSF
0,491:00.004
0.502:20.008
0.347:000028
0.202-00002
0.532::0.014
0.580:0.0029
0.329-0.000
0.205
20

---

Ablation Study of CondTSF
We compare the changing trajectories of error during the dataset condensation process. Since MTT has been proven to be suitable backbone for CondTSF, we conduct experiments on different methods of plugging CondTSF into MTT. We utilize the standard condensing ratio as shown in T
Test error calculated such. After the synthetic data has been distilled, it is used train randomly initialized testing models. After training with the synthetic dataset, the models are tested on the test sampled from the source dataset. MAE error is reported in the figures below.
F.1 Performance of CondTSF with Different Gap
HIT
HTT spad Credial upad
HIT
HTT upacc Ciddt1
Contial Eipac
ConTEFF ipad
Toal Error
Teel Error
0.C-
80
110
190
310
11o
300
80
110
199
300
11o Condensation Epocits
Condensaioo Epochs
Condensation Epochs
Condensation Epochs
ExchangeRate
(b) Weather
Electricity
(d) Traffic
HITT
HITT
HTT
NITT
tepach
Teal Eares
Teal Errer
0.5
0.5
10t Condensation Epochs
130
130 Condensation Epochs
310
200
120 Condensation Epochs
Condensation Epochs
(e) ETTml
(f) ETTm2
ETTh1
(h) ETTh2
Figure 5: Yellow: Use MTT to distill for 200 epachs. Orange: Use MTT to distill for 200 epochs and use CondTSF to update in every epoch.
HTTT
HTT NITO gods Crddaa spac
HTT NTT3 qudda CiddTS upad
NTT NIT spacia CodfTS spad
gode Ceddi lspaa
Teat Erpor
Teel Ereur
0.1
Vellowe Agein
13n
140
311
110
300
110
199
300
11
300
Condensation Epochs
Condensation Epochs
Condensation Epochs
Condensation Epocis
(a) ExchangeRate
(b) Weather
(c) Electricity
(d) Traffic
METT
HIT
IITT
NIT
Test Exror
Teal Eerur
0.5
120
100
280
120
300
30
120
190
280
120
200
Condensation Egochs
Condensation Epochs
Condensation Epochs
Condensation Epochs
(e) ETTml
(f) ETTm2
ETTh1
(h) ETTh2
Figure 6: Yellow: Use MTT to distill for 200 epochs. Orange: Use MTT to distill for 200 epochs and use CondTSF to update every 3 epochs.
21

---

NTT
NTT
NTT
MTT
lipeh
rpad
Teal Earee
Tost Earer
Teal Error
0.7
10t
120
330
19t
200
12B 310
Condensation Epochs
Condensation Epochs
Condensation Epochs
Condensation Epochs
(a) ExchangeRate
(b) Weather
(c) Electricity
(d) Traffic
Taat Erout
6.
Test Eenur
0.3
0.5
40 50
13n
160
306
8
12n
190
300
30 Condensation Epochs
120
16t
280
43
120 141
30
Condensation Epochs
Condensation Epochs
Condensation Epochs
(c) ETTml
(f) ETTm2
ETTh1
(h) ETTh2
Figure 7: Yellow: Use MTT distill for epochs. Orange: Use MTT to distill 200 epochs use CondTSF to update every epochs.
We observe that CondTSF consistently reduces the testing error with different utilization gaps.
F.2 Relationship of Performance and Label Error
We also conduct experiments on label error and test error. We visualize the trajectory of label error Clabel and test error through the distillation process. The results shown in Fig 8
Model 1: Use MTT to distill for 200 epochs.
Model 2: Use MTT to distill for 160 epochs and then use CondTSS to update for 40 epochs.
As shown in Fig it can be observed that using MTT leads increase in label error Ltobel- While applying CondTSF effectively lowers the label error in the last 40 epochs, and therefore enhancing the performance.
22

---

CondTSF's used
Con is used
0.6
Test Error of Model
0.8
Test Error of Model
Errur Model
Label Error
0.7
Label Error
100
0.5
Test Error
0.6
75
40
0.5
50
0.4
0..4
Label Error of Model
0.3
25
Label Error of Model 1
Model
Label Error of Model
0-
40
80
120
169
200
0
40
120 Condensation Epochs
80
160
200
Condensation Epochs
(a) ExchangeRate
(b) Weather
CondTSF used
Cond
0.9
Test Error 1
Test Error of Model
150
Test Error of Model
0.8
Text Error of Model
0.9
Label Error
0.7
Label Error
0.8
100
0.6
0.7
0.5
50
0.6
Labe Model
0.4
Ereor Model
0.5
Label Error of Model
Label Error of Model
0.3
0.4
Condensation Epochs
Condensation Epochs
(c) Electricity
(d) Traffic
Con used
CondTSF is used
0.8
0.8
150
Errue mm
Test Error Model
Test Error Model 2
40
Test Error Model
Label Error
100
0.7
Test Error
Label Error
30
0.5
20
Test
50
0.4
10-
Jabel Error Model
Label
0.3
Label Error Model
0.5
Label Error Model
0
0.2
80
120
160
200
40
80
120
160
200
Condensation Epochs
Condensation Epochs
(e) ETTml
(f) ETTm2
CondTSF used
Cond is used
200
1.0
Test Error Model 1
Test Error of Model
150
Test Error Model
Label Error
Test Error
Label Error
150
0.8
Test Error
100
0.7
100
0.6
50-
0.6
50
0.5
Label Error of Model
Lahel Error of Model
Label Error of Model 2
0.5
Label Keror of Model
0.3
8o Condensation Epochs
120
160
200
40
80
120
160
200
Condensation Epochs
gg) ETThl
(h) ETTh2
Figure 8: Visulization of the training curve label error and test error during the distillation process.
23

---

G Parameter Sensitivity of CondTSF
We CondTSF with different update gaps G and additive update ratios We utilize the standard distill ratio as shown in Table Our observations indicate that CondTSF displays a notable degree of robustness concerning these parameters. Specifically, the effectiveness CondTSS persists when the update gap G' is moderately sized and additive update ratio 3 is not excessively small.
030
62)
825
421
4as sing Can Addavi Updane
Aaw Rine Fing Satp Aditwwe Updse
Cips of o o
Gaps of Courrol
ExchangeRate
(b) Weather
6.3
641
Caye Usde Coners
of cunerss
Addivve ppost Rodm
6.22
(c) Electricity
(d) Traffic
A25
Gags Usig CHUTs?
Ray Adoive Updete Roate
Caps Usig coorto
Row Addin Updee Reste
(c) ETTml
(f) ETTm2
$.55
550
Gyye of Uady ciatt
Lig Addave Updete baain
Gape of Courrod
Lage Addowe Updase Roder
ETTh1
ETTH2
Figure 9: Performance of CondTSS with different update gaps and update ratios.
24

---

H
Visualization of Synthetic Data
We provide some visualization of synthetic data distilled by MTT and MTT+CondTSF all datasets. It is observed that the synthetic dataset distilled with CO is smoother than the ones without CondTSF. Smoother data indicates more generalized features and therefore helps boost the performance.
ETTml
ETTm2
ETTh
ET1b2
11
Bundom
Ma
Pw
50
20
CoddINF
106
Eachangsltate
Weather
Electiisty
Traffic
Bandom
60
35
26
nana
mM
WM
26
15
Figure 10: Visualization of synthetic data.
rww h
MM
Random Init
Ma
MIT
CondTNF
Exchangeltate
Weather
Klectrictty
Random Init
20
20
20
MTT
Mmw
Www.
20
60
75
100
20
60
CoddTSP
20
20
Figure 11: Visualization of synthetic data.
25

---

ETTml
KTTm2
KTTbl
ET152
M
ExchanacRate
Weatber
Electridit
Mana
Suy
CoddIN
Figure Visualization of synthetic data.
ml
ETTW2
KTIM
E1133
MAM
Randem Init
You
16
26
Mas
MMy
Yerh wm/h M
MTT
NW
20
28
CoadTSF
56
300
26
Rachangsltate
Weather
Klsctiiity
Traffic
Wy
Randem Init
26
1
16
100
M
M
MIT
vma
mm
MN
21
20
16
90
15
190
21
60
66
CoddSS
MNN
26
08
15
100
20
16
Figure 13: Visualization of synthetic data.
26

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
As shown in Sec we explicitly claim the contributions of this work.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
As shown in Sec we discuss the limitations of this work.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
27

---

Justification:
We propose two theorems in Sec We provide complete proof of the two theorems in Apple A.I and A.p
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide the detailed algorithm in Sec We provide detailed parameter settings to reproduce the results in Sec 5
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
28

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We attach the code needed to reproduce the results with the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ praips.cc/ubblic/guuueee CoodeSumeeieeeeee ssi iessonPolll mollicyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide detailed hyperparameter settings in Sec that are necessary to reproduce the results.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We repeat each experiment five times and the average and standard deviation of emrors are reported in Sec
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
29

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
As shown See we provide the model the GPU we used for experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make it into the paper).
9. Code Of Ethics
Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics htttps///beurrpp..cc///bbbb//att
Question:
Answer: [Yes]
Justification: This work does not involve human subjects. shown See 5 the datasets used in this work are all widely used open-source datasets.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work does not involve human subjects. As shown Sec the datasets used in this work are widely used and open-source. There's harm to humans leak of privacy in this work.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
30

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: As shown in Sec the datasets used in this work are all widely used and open-source. There's no risk of releasing unsafe data.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: As shown in Sec we cite the papers that produced the code. The datasets used in this work are all widely used and open-source.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
31

---

If assets are released, the license, copyright information, and terms of use in the thcode has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
latasets.paperssi
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documents with the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/codeemoodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
32

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
33


Reviews:
[{"review_id": "NteIAL6fPq", "reviewer": "Reviewer_ivQc", "summary": "This paper provides a simple fix on the parameter-matching based dataset condensation methods for time series forecasting tasks. Besides matching the parameters of the model $M_f$ trained on the full training set and $M_s$ trained on the smaller synthetic dataset, it additionally induces the $M_f$ to perform well in the synthetic data too. It theoretically shows that the additional regularization contributes to an surrogate upper bound of the original objective, and empirically proves that the fix help improve various dataset condensation methods.", "questions": "1. In line 162, the author claims that it is \"observed\" that models initialized with different samples of parameter values predict similarly after trained on the full dataset given arbitrary input. Is it supported by any evidences, either theoretically or empirically?\n2. Some clarity on optimizing gradient term can be provided. For example, when the parameters are matched and $M_s$ are well fitted, why the label error of $M_f$ on $s$ will increase?", "limitations": "The limitation that the analysis only applies to linear models is discussed in the paper.", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written with clear structure and proper organization, and the motivation of the proposed condTSF plugin is solidly grounded by theoretical analyses\n2. The proposed method is simple and easy to implement. \n3. Impressive improvement are observed in the experiments on various baselines and benchmarks.", "weaknesses": "A few details are not clear. See questions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**Q1:**\nIs the claim \"models initialized with different samples of parameter values predict similarly after trained on the full dataset given arbitrary input\" in line 162 or Eq(9) supported by any evidence?\n\n**A1:**\nWe apologize for the ambiguity. Please refer to Figure 3 and line 166-168, we support this empirical observation with experiments. The large orange points in the figure are input data, and the yellow and blue points are predictions of the models initiated with different parameters. We used MDS algorithm to reduce the dimension of the original prediction and data. MDS algorithm preserves the distance between points, which means points closer to each other in high dimensions are also close in low dimensions. It can be observed that the predictions of the models are similar despite their initial parameters are different.\n\n----\n\n**Q2:**\nWhy the label error on $s$ will increase when the parameters of $M_s$ are matched.\n\n**A2:** We apologize for the ambiguity.\n- Firstly, The label error is used to evaluate the synthetic dataset $s$ instead of the expert model $M_f$. Please refer to Thm.2 and Eq(11), label error is an upper bound of the **value term**. Therefore label error ensures the similarity between prediction values of $M_f$ and $M_s$.\n- Secondly, the parameter loss is an upper bound related to the **gradient term**. It only ensures $M_s$ and $M_f$ have similar gradients on the input data, yet it does not ensure their value of predictions are similar.\n- To sum up, our analysis in Sec.4.1 and Thm.1 showed that **value term** and **gradient term** are decoupled in the optimization. Parameter loss relates to the upper bound of **gradient term** while label error is the upper bound of **value term**. Therefore, even when parameter loss is small, the label error can still be large.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uOFdekzJAk", "reviewer": "Reviewer_TXkn", "summary": "In this paper the authors study the question of dataset condensation / distillation in the context of time series forecasting. They propose a decomposition that could be used to empirically estimate bound the point-wise forecast MSE loss. Based on this decomposition the authors proposed a condensation plug in on top of existing data condensation method that updates the distilled dataset with the estimated full model forecast. Through empirical study they show that with this plug in the resulting condensed dataset leads to better models across different model classes and benchmark forecasting tasks.", "questions": "Besides the questions in the weakness:\n\n- While the distill ratio in Table 2, e.g. condensing the datasets into one training example, can be used to show the extremity of the distillation process,  it is questionable what this one-shot learning means for the MLP, LSTM, CNN involved. Any insights, especially why to choose them over the few-shot study in, e.g. Appendix F (this one has visualization as well) to be in the main paper? \n- As now the expert model is more actively involved through the two decomposition steps to touch the synthetic data, how does the capacity of the expert impact the fidelity of the synthetic dataset versus the full one? What about at least one non linear expert just to get some insights?\n- The method proposed here does not utilize any time series concepts. If so this is an application on multivariate regression with linear expert. What's the novelty beyond that for time series exclusively?", "limitations": "The authors addressed the limitations.", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 3, "strengths": "The paper attempts a new and simple solution to an important training data efficiency problem. It also attempts a novel prospect of analyzing the dataset distillation through the loss decomposition.", "weaknesses": "The major weakness is the writing. It is hard to follow in its current shape, in particular the math description of the research question should be more rigorous. To name a few:\n- What are the sample space of x and s? They now look like a single univariate time series. It is the case?\n- What is t' in Eqn (5)? Is it summed over? If so is it double sum within sum_t?\n- How is Eqn (7) and (8) optimized (also in Algo 1).\nSimilarly regarding writing, the paper does not motivate the research question well, and has not provided enough background details on its base distillation methods (Line 200 Model Settings), especially MTT(3). As a result it is hard to evaluate the novelty of the paper when the main idea is not clearly delivered.\n\nThough mentioned as a limitation, the assumption of linear model trivializes the theories, which ties the proposed method to DLinear expert, and makes it hard to understand how the learning from the synthetic data generalizes beyond the expert's model class. There are similar simplifications, e.g. the empirical approximation as in Eqn (9) that's observational.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We apologize for the ambiguity.\n\n**W1:** What are the sample space of $x, s$? Are they univariate time series?\n\n**A1:** Yes, $f, x, s$ are univariate time series. Please refer to Sec.3, l.93-95. A time series dataset can be viewed as a long vector. We cut the dataset into two vectors and obtain train set $f$ and test set $x$. Both $x$ and $f$ are vectors. Then we sample a short continuous part of train set $f$ as the initial synthetic dataset $s$. $s$ is still a vector.\n\n---\n\n**W2:** What is $t'$ in Eq(5)? Is it summed over?\n\n**A2:**\n- We use $s_{t':t'+m}$ to denote a vector(part of $s$) with length $m$ starting from position $t'$. $t'$ is arbitrarily chosen.\n- Please refer to Eq(16) and l.444-446, $s_{t':t'+m}$ is an arbitrary vector used to perform Taylor expansion to get the value of $M(x_{t:t+m})$. The idea throughout our paper is that test set $x$ is unavailable during dataset condensation, thus we need to substitute $x$ with accessible synthetic dataset $s$ so the optimization objective becomes optimizable.\n- The sum in Eq(5) is only performed on $t$, not $t'$. Please refer to Eq(3) and App.A1, $\\sum_t$ is used in test error to sum over the test set $x$. For each $x_{t:t+m}$, we use an arbitrary $s_{t':t'+m}$ to perform Taylor Expansion, thus $t'$ is not summed over.\n\n----\n\n**W3:** How are Eq(7) and(8) optimized? Lack of information on base methods and the novelty is not clear.\n\n**A3:**\n- Eq(7) and(8) are optimized the same way as MTT[1], which is a well-acknowledged dataset condensation method. According to our analysis, MTT is optimizing the **gradient term**. There are multiple dataset condensation methods that use MTT as the backbone, such as TESLA[2], FTD[3], DATM[4], etc. We will add introduction of MTT in appendix in the next version.\n- MTT and all other backbone methods are designed for classification. They perform poorly on time series forecasting. The novelty of our method is that CondTSF enhanced the performance of all previous methods on time series forecasting.\n\n[1] Dataset Distillation by Matching Training Trajectories\n\n[2] Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory\n\n[3] Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation\n\n[4] Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching\n\n---\n\n**W4.1:** The hypothesis of linear model trivialized the work.\n\n**A4:** \n- Please refer to App.A1 and Eq(16), using nonlinear models will add 2nd and higher order terms to the Taylor Expansion. We ignore higher order terms because their effect on the output is minor. Meanwhile, although linear models are simple in structure, they perform well on time series forecasting and outperform complicated transformer models[1].\n- We add experiments on using nonlinear models as expert models, please refer to **A7**.\n\n[1] Are Transformers Effective for Time Series Forecasting?\n\n---\n\n**W4.2:** The observation in Eq(9) is a simplification.\n\n**A5:** The observation in Eq(9) is not a simplification. Please refer to l.157-159, The test model parameter is unknown during dataset condensation, thus we need to substitute them with accessible parameters. Thus Eq(9) is necessary, otherwise the **value term** is not optimizable.\n\n---\n\n**Q1:** Why is the result of one-shot dataset for models MLP, LSTM, CNN presented in the main text?\n\n**A6:** Cross-architecture performance is an important metric reported in all dataset condensation papers. MLP, CNN and LSTM perform well on time series forecasting[1] and are widely used in the evaluation of dataset condensation[2]. Thus we present the result in the main text.\n\n[1] Are Transformers Effective for Time Series Forecasting?\n\n[2] Dataset Condensation for Time Series Classification via Dual Domain Matching\n\n---\n\n**Q2:** Add a nonlinear expert?\n\n**A7:** Please refer to App.A1 and Eq(16). Given nonlinear models, our method ignores the 2nd and higher order terms in Taylor Expansion. We conduct experiments to show that ignorance is acceptable. We use nonlinear models CNN, 3-layer MLP(ReLU activated) as expert models. Average MAE of 5 models is reported below. The results show that CondTSF performs well with nonlinear models.\n\n- CNN\n\n||Ex.Rate|Wea.|Elec.|Tra.|ETTm1|ETTm2|ETTh1|ETTh2|\n|-|-|-|-|-|-|-|-|-|\n|MTT|0.372|0.314|0.482|0.662|0.550|0.347|0.644|0.371|\n|MTT+CondTSF|0.140|0.246|0.357|0.451|0.482|0.237|0.460|0.297|\n|TESLA|0.378|0.310|0.516|0.655|0.544|0.359|0.634|0.365|\n|TESLA+CondTSF|0.134|0.253|0.374|0.528|0.499|0.251|0.473|0.293|\n|DATM|0.331|0.335|0.504|0.587|0.566|0.318|0.633|0.349|\n|DATM+CondTSF|0.137|0.291|0.355|0.452|0.518|0.231|0.451|0.290|\n\n- 3-layer MLP(ReLU activated)\n\n||Ex.Rate|Wea.|Elec.|Tra.|ETTm1|ETTm2|ETTh1|ETTh2|\n|-|-|-|-|-|-|-|-|-|\n|MTT|0.364|0.311|0.475|0.633|0.564|0.362|0.615|0.354|\n|MTT+CondTSF|0.139|0.248|0.375|0.501|0.493|0.245|0.423|0.285|\n|TESLA|0.352|0.297|0.525|0.594|0.541|0.341|0.606|0.337|\n|TESLA+CondTSF|0.128|0.252|0.397|0.488|0.487|0.250|0.438|0.283|\n|DATM|0.326|0.349|0.517|0.622|0.558|0.329|0.593|0.350|\n|DATM+CondTSF|0.141|0.254|0.385|0.496|0.498|0.234|0.437|0.286|\n\n---\n\n**Q3:** What's the novelty beyond time series?\n\n**A8:** \n- All previous dataset condensation methods based on classification perform poorly on regression. The core novelty of our method is that it is the first dataset condensation method focusing on regression and significantly enhances the performance of all backbone methods.\n- In time series forecasting, when generating training data, the data is usually sampled overlap from the dataset. For instance, the data points can be sampled as $x_{0:24}, x_{4:28}$, etc. Please refer to Eq(12), inspired by the overlap property, we utilize an additive method to gradually update the synthetic data instead of directly using $M_f(s_{t':t'+m})$ to overwrite $s_{t'+m:t'+m+n}$ to avoid vibrations.\n- Additionally, other content related to time series can be explored in future works. We presented a simple yet effective method, which can serve as a prototype.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ekR7WfuvQq", "reviewer": "Reviewer_DVXT", "summary": "This paper explores dataset condensation, which generates a small dataset for training of deep neural networks, for time series forecasting. Specifically, it proposes a one-line dataset condensation plugin designed specifically for time series forecasting. It first formulates the optimization objective of dataset condensation for time series forecasting, and based on which a reformulation is proposed and further theoretical analysis is performed. Based on the proposed theoretical analysis, this paper proposes a simple data condensation plugin CondTSF. Experiments on 8 time series datasets are performed to demonstrate the effectiveness of the proposed CondTSF.", "questions": "Please refer to the weakness part.", "limitations": "Please refer to the weakness part.", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. It proposes a simple dataset condensation plugin. \n2. A theoretical analysis is provided.", "weaknesses": "1. Dataset condensation is a strategy specifically designed for training large dataset. However, the data size in time series is usually significantly smaller than that in NLP and CV. In my experience, the computational demand to deal with time series data may not be an urgent request in real-world time series applications. The topic discussed in this paper seems interesting given the current program of different foundation models, but I highly doubt it applicability in the field of time series. \n2.  A related problem is that how the time series foundation model performs remain unclear. Exploring the foundation model for time series may be a promising direction, but it is not widely accepted and deployed in real-world applications. In fact, considering the semantics of time series is highly dependent on the underlying applications, so far I have not observed any foundation model can provide good zero-shot learning capability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "**W1:** The demand for dataset condensation for time series forecasting is not urgent. What are the real applications of dataset condensation for time series foundation models?\n\n**A1:**\n- Firstly, dataset condensation is also important in other aspects besides reducing computational cost. For instance, data condensation is proven to benefit downstream tasks such as continual learning[1], privacy[2], etc. These tasks are vital for some aspects of the time series, such as finances.\n- Secondly, in the era of large models, Neural Architecture Search(NAS) of large time series foundation models is still computationally intensive[3]. Utilizing condensed data to identify a smaller parameter set and conducting full training on this set can improve efficiency.\n- Moreover, our proposed method could serve as a prototype and shed light on the related research area.\n\n[1] An Efficient Dataset Condensation Plugin and Its Application to Continual Learning\n\n[2] Privacy for free: How does dataset condensation help privacy\n\n[3] Dataset Condensation for Time Series Classification via Dual Domain Matching\n\n----\n\n**W2:** How the time series foundation model performs remains unclear, and most are not good at zero-shot learning.\n\n**A2:** \n- Indeed, nowadays foundation models do not behave well in time series forecasting[1]. \n- However, more explorations of foundation models are carried out[2]. Therefore, our dataset condensation method can be used to lower the computational cost and accelerate the exploration of foundation models on time series tasks[3][4][5].\n- Meanwhile, our proposed method points out an interesting direction in time series forecasting tasks. More efforts can be devoted to this field and our method can serve as a prototype.\n\n[1] Are Language Models Actually Useful for Time Series Forecasting?\n\n[2] A Survey of Time Series Foundation Models: Generalizing Time Series Representation with Large Language Model\n\n[3] Time-llm: Time series forecasting by reprogramming large language models\n\n[4] Timegpt-1\n\n[5] A decoder-only foundation model for time-series forecasting", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "boxlkP5tFE", "reviewer": "Reviewer_s2gY", "summary": "In this paper, the authors propose a dataset distillation approach by decomposing the distillation loss into two terms, the value term and the gradient term. Then, they derive bounds for these two terms that enable more effective dataset distillation for time-series forecasting. Strong performance is demonstrated in the conducted experimental evaluation", "questions": "Please respond to the comments provided in the drawbacks. Overall, strong performance is demonstrated, but there are several aspects that should be improved.", "limitations": "Yes.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "+ The paper is well-written and easy to follow.\n\n+ Authors attempt to support their methodology through theoretical explanations\n\n+ Strong performance is obtained in the experimental evaluation.", "weaknesses": "- p4. l121. The authors do not explain why this non-optimizable $\\epsilon$ error occurs. \n\n- l. 137 in principle we could directly optimize this gradient term without deriving any bound. Unless I am missing something, this is supported by DL frameworks (e.g., PyTorch). On the other hand, I understand that there might be computational benefits from deriving a bound that does not involve the gradients. However, authors should justify their choice (why we need to derive this bound) and if this is related to computational efficiency. Ideally, experiments should be provided to support their choice. \n\n- At various points there are implicit assumptions the effect of which is not appropriately discussed, e.g., Eq. (9) is essentially supported by an experimental result in a Figure and a verbal description. I do not (empirically) disagree with the conclusion, but I can see cases where this would not hold (e.g., in financial time-series forecasting, where using the same data we can have vastly different (non-linear) hypotheses). Whether this alters the conclusions/and or the method, is not appropriately discussed.\n\n- The updating process in (12) essentially moves the targets towards the predictions of the model. This is a reasonable result that is well-known to work. However, the way this is applied is now very close to target smoothing/model distillation approaches that essentially do the same thing (also applied in the context of time series forecasting as far as I know). There is some literature on model distillation approaches (and/or self-distillation) in the context of time-series forecasting and/or regression, which essentially demonstrate that this kind of \"smoothing\" can indeed increase performance. Therefore, this raises some questions on whether the method should be compared experimentally (or at least theoretically) with such approaches.\n\n- Even though some ablation studies were provided in the Appendix, I didn't manage to find perhaps the most important result: What is the impact of the gradient term and what is the impact of the value term? What would happen if we remove the L_param loss in l. 9 of Alg 1? What would happen if we removed the value term?\n\n- Isn't the method also applicable in the case of regression tasks? If I am not missing anything, I think the same methodology without any change (apart from the input to the models) can be applied also to handle generic regression tasks that also expand over time-series forecasting.\n\n\nMinor: l.147-152: I think $k$ is used before being introduced.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We apologize for the ambiguity.\n\n**W1:** What does non-optimizable term $\\epsilon$ mean?\n\n**A1:** \n- Please refer to Eq(4), since test label $x_{t+m:t+m+n}$ is unavailable during dataset condensation, we transform it to the prediction of an expert model $M_f$ on test data $x_{t:t+m}$ for further analysis. Since there is always an error between the prediction of the model and the ground truth, we use $\\epsilon$ to denote it.\n- $\\epsilon$ is optimized during the training process of expert model $M_f$. However, since we can only have access to the trained expert model $M_f$ during dataset condensation, $\\epsilon$ is no longer optimizable.\n\n---\n\n**W2:** Why derive a bound of the gradient term?\n\n**A2:**\n- Indeed, the original form of **gradient term** in Eq(5) is not directly optimizable. Test data $x_{t:t+m}$ exists in **gradient term**, which is unavailable during dataset condensation.\n- To solve this problem, please refer to Eq(6), we use Cauchy-Schwarz Inequality to decouple **gradient term**. Then we get the optimizable part $||\\theta_f-\\theta_s||^2$(the gradient of linear model is the parameter), and the non-optimizable part $||x_{t:t+m}-s_{t':t'+m}||^2$.\n- Thus, it is necessary to derive the upper bound of **gradient term** because the original one is not optimizable.\n\n---\n\n**W3:** How to justify the empirical finding?\n\n**A3:** In the cases you mentioned, the model is theory-driven, which means the prediction of the model relies highly on the hypotheses and theories. However, in this case, the models are data-driven, and no additional hypotheses are made. The prediction of the model relies only on the training data and the model structure. Thus, in this case, if the models share the same architecture and are trained with the same dataset, empirically they output similar outputs.\n\n---\n\n**W4:** How does our method compare to smoothing or model distillation?\n\n**A4:** \n- Firstly, model distillation and dataset condensation are different tasks. Model distillation updates student model parameter and produces distilled model. Dataset condensation updates the synthetic data and produces distilled dataset.\n- Secondly, model distillation aims at aligning the output of $M_s$ and $M_f$. The training of $M_s$ is directly supervised by $M_f$, where $L$ is the loss function, e.g. MSE.\n  $$M_s=\\arg\\min_{M} L(M(s),M_f(s))$$\n  However, in the case of our **value term** optimized by Eq(12), $M_s$ is trained alone using synthetic data $s$ and $M_f$ only has indirect access to the training of $M_s$.\n  $$s=\\arg\\min_s L(M_s(s),M_f(s))$$\n  $$\\textbf{s.t.}\\hspace{1em} M_s=\\arg\\min_M L_{train}(M, s)$$\n  Thus model distillation is not comparable to our method.\n- Meanwhile, we conduct experiments using LPF to perform target smoothing on the synthetic data and compare the results with CondTSF. Average MAE of 5 models is reported below. It shows that CondTSF is significantly more effective than target smoothing.\n\n||Ex.Rate|Wea.|Elec.|Tra.|ETTm1|ETTm2|ETTh1|ETTh2|\n|-|-|-|-|-|-|-|-|-|\n|MTT|0.778|0.509|0.747|0.742|0.653|0.685|0.693|0.719|\n|Smooth|0.867|0.602|0.831|0.786|0.728|0.706|0.701|0.744|\n|MTT+Smooth|0.620|0.501|0.788|0.836|0.656|0.619|0.680|0.704|\n|MTT+CondTSF|0.195|0.326|0.391|0.494|0.491|0.347|0.532|0.329|\n\n---\n\n**W5:** What if the value term or the gradient term is removed? What are the impacts of these two terms?\n\n**A5:** $L_{param}$ is the same form of Eq(8) in Sec.4.2, which is used to optimize **gradient term**. Please refer to l.143-145, our analysis shows that previous dataset condensation methods based on parameter matching, e.g. DC[1], MTT[2], etc. are optimizing the **gradient term**. Our method CondTSF is optimizing the **value term**.\n- If **value term** is removed: The method falls back to the previous dataset condensation methods based on parameter matching(DC, MTT, etc.), and the results can be found in Table.1,3,5,7,9. Moreover, please refer to Fig.8, in the beginning 160 epochs, only **gradient term** is optimized, in the last 40 epochs, **value term** is also optimized. Optimizing **value term** significantly enhances the performance.\n- If **gradient term** is removed: Please refer to l.222-224 and App.C, experiments on backbone methods that are not based on parameter matching, e.g. DM[3], IDM[4], etc. are also carried out. These backbone methods do not optimize the **gradient term**. Thus when combining these backbone methods with CondTSF, only **value term** is optimized, and the results are in Table.8. We conduct experiments that only optimize **value term** without any backbone methods, average MAE of 5 models is reported. The results are well-aligned with our analysis that optimizing **gradient term** or **value term** alone harms the performance.\n\n||Ex.Rate|Wea.|Elec.|Tra.|ETTm1|ETTm2|ETTh1|ETTh2|\n|-|-|-|-|-|-|-|-|-|\n|MTT (only gradient term)|0.778|0.509|0.747|0.742|0.653|0.685|0.693|0.719|\n|CondTSF (only value term)|0.563|0.537|0.761|0.809|0.738|0.538|0.623|0.569|\n|MTT+CondTSF|0.195|0.326|0.391|0.494|0.491|0.347|0.532|0.329|\n\n[1] Dataset Condensation with Gradient Matching\n\n[2] Dataset Distillation by Matching Training Trajectories\n\n[3] Dataset Condensation with Distribution Matching\n\n[4] Improved Distribution Matching for Dataset Condensation\n\n---\n\n**W6:** Is this method applicable to all regression tasks?\n\n**A6:** Our proposed CondTSF is applicable to generic regression. Meanwhile, we have designs that are related to time series forecasting. In time series forecasting, when generating training data, the data is usually sampled overlap from the dataset. For instance, the data points can be sampled as $x_{0:24}, x_{4:28}$, etc. Please refer to Eq(12), inspired by the overlap property, we utilize an additive method to gradually update the synthetic data instead of directly using $M_f(s_{t':t'+m})$ to overwrite $s_{t'+m:t'+m+n}$ to avoid vibrations.\n\n---\n\n**Minor:** $k$ in l.147-152 used before introduced.\n\n**A:** We are sorry for the ambiguity. We will correct that in next version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
Dear reviewers and area chairs,

We thank all reviewers and area chairs for their valuable time and comments.

We have responded to each reviewer individually to address any comments. We would like to give a brief summary.

1. We clarify some of the meanings of the notations in our paper to address the concern.
2. We provide information on backbone methods and explain the novelty of the work. We also explain that our proposed method has designs that relate to the features of time series.
3. We add experiments using nonlinear models as expert models and show that CondTSF also works well with nonlinear expert models.
4. We explain the practical applications of dataset condensation and show that dataset condensation for time series forecasting has pragmatic values.
5. We show that deriving the upper bound of the **gradient term** is necessary because there exists unavailable test data in the original form.
6. We show that **dataset condensation** and **model distillation** are distinct tasks, and thus the methods are not comparable.
7. We add experiments using a low pass filter to perform target smoothing on synthetic data and show that CondTSF is significantly more effective than target smoothing.
8. We add experiments using only CondTSF(optimizing only **value term**) and show the impacts of **gradient term** and **value term**.

Again, we thank all reviewers and area chairs!

Best,

Authors

Meta-review: This paper introduces a simple and effective method for condensing time series datasets, which can be implemented as a one-line plugin for time series forecasting models corresponding to an additional regularization term.
The paper develops a theory showing that this regularization term allows for upper bounding the objective of the dataset condensation problem.
The resulting algorithm is then empirically validated on 8 time series datasets which confirm the effectiveness of the proposed method.
The reviewers generally agree that the paper is clearly organized and structured. Other strengths of the paper include the simplicity of the resulting method, the theoretical analysis, and the strong performance in experiments. However, some reviewers raised concerns about the ambiguous notation, the lack of motivation and significance of addressing dataset condensation for time-series, and the limited evaluation.
The rebuttals satisfactorily addressed most of these concerns, including adding additional experiments to verify that the methods is applicable to nonlinear models, and including comparisons against more baselines like target smoothing on synthetic data.
Overall, the reviewers found the paper to be a solid enough contribution to warrant acceptance. Reviewers however recommend that acceptance be conditional on the additional results in the rebuttals and notation and presentation improvements be included and prominently showcased in the camera-ready version of the paper.
Decision: Accept (poster)