Using Synthetic Data to Improve the Long-range Forecasting of Time Series Data
Authors: Shiyu Liu
Mehul Motani
Venue:  (2021)
Keywords: long-range time series data prediction
Generative Adversarial Network
Long Short-term Memory

Abstract: Effective long-range forecasting of time series data remains an unsolved and open problem. One possible approach is to use generative models to improve long-range forecasting, but the challenge then is how to generate high-quality synthetic data. In this paper, we propose a conditional Wasserstein GAN with Gradient and Error Penalty (cWGAN-GEP), aiming to generate accurate synthetic data that preserves the temporal dynamics between the conditioning input and generated data. By using such synthetic data, we develop a long-range forecasting method called Generative Forecasting (GenF). GenF consists of three key components: (i) a cWGAN-GEP based generator, to generate synthetic data for next few time steps. (ii) a predictor which makes long-range predictions based on generated and observed data. (iii) an information theoretic clustering (ITC) algorithm to better train the cWGAN-GEP based generator and the predictor. Our experimental results on three public datasets demonstrate that GenF significantly outperforms a diverse range of state-of-the-art benchmarks and classical approaches. In most cases, we find an improvement of at least 10% over all studied methods. Lastly, we conduct an ablation study to demonstrate the effectiveness of the cWGAN-GEP and the ITC algorithm.

Under review as a conference paper at ICLR 2021
USING SYNTHETIC DATA TO IMPROVE THE LONG- RANGE FORECASTING OF TIME SERIES DATA
Anonymous authors
Paper under double-blind review
ABSTRACT
Effective long-range forecasting of time series data remains unsolved and open problem. One possible approach is to use generative models to improve long-range forecasting. but the challenge then how to generate high-quality synthetic data. In this paper, propose conditional Wasserstein GAN with Gradient and Error Penalty (cWGAN-GEP), aiming to generate accurate synthetic data that preserves the temporal dynamics between the conditioning input and generated data. By using such synthetic data, we develop a long-range forecasting method called Generative Forecasting (GenF). GenF consists of three key components: (i) &WGAN- based generator, to generate synthetic data for next few time steps. (ii) a predictor which makes long-range predictions based generated and observed data. (iii) information theoretic clustering (ITC) algorithm to better train the eWGAN-GEE based generator and the predictor. Our experimental results on three public datasets demonstrate that GenF significantly outperforms diverse range of state-of-nd tate--of-hhe-ar benchmarks and classical approaches. In most cases, we find an improvement of at least 10% over all studied methods. Lastly, we conduct an ablation study to demonstrate the effectiveness of the eWGAN-GEP and the ITC algorithm.
INTRODUCTION
Short-range forecasting of time series data has been able to provide some useful information, but its scope of application is limited (Chatfield 2000) Granger & Newbold, (014). In most applications, long-range forecasting of time series data is preferred as it allows more time for early intervention and
planning opportunities (Alvarez et 2010 Azad al. 2014). For example, long-range forecasting of patient's vital signs effectively gives clinicians more time to take action and may reduce the occurrence of potential adverse events (Luo 2015 Choi & et al 2016).
However, a critical issue affecting the long-range forecasting is that the predictive performance (c.g.. N-step ahead) becomes worse N grows. One practical approach address this problem is to use synthetic data to shorten the prediction horizon N. For example, in iterative forecasting (Marcellino) al. 2006) Hamzzçębi al., 2009), the previous predictions are used together with the original data as new inputs to evaluate the next prediction. However, the synthetic data (i.e., previous predictions) generated in this recursive and supervised manner more susceptible to error propagation (Sorjamaa) & et al 2007 Weigend & Gershenfeld] 2018). Overall, the quality of synthetic data is the key to improving long-range forecasting. recent years, the success of Generative Adversarial Network (GAN) (Goodiellow & et al 2014) in replicating real-world content has inspired numerous GAN based architectures (Frid-Adar et al. 2018a by Shin et al. 2018 Bowles et al.) 2018). in order to generate synthetic data for different purposes (e.g., improve classification accuracy). However, the utilization of synthetic data to improve long-range forecasting remains unexplored.
In this paper, we contribute the area of long-range forecasting as follows.
1. We augment the existing conditional Wasserstein GAN with Gradient Penalty with a mean squared error term. This new architecture, called eWGAN-GEP, aims to generate accurate synthetic data which preserves the temporal dynamics between the conditioning input and generated data.
2. We develop a long-range forecasting method called Generative Forecasting (Genl) which consists of three key components:
(i) a eWGAN-GES based generator, to generate synthetic data for next few time steps. (ii) a predictor which makes long- rongerange predictions based on generated and observed data. (iii) an information theoretic clustering (ITC) algorithm to better train the eWGAN-GEI based generator and the predictor.

---

Under review as a conference paper at ICLR 2021
X
X2
XM
XM+1
XM+L
XM++N
Time Step
Observation Window (Length = M) Synthetic Data Window (Length L) Prediction Horizon (Length=N)
Figure I: Long-range forecasting via observation//syatetic data window and prediction horizon.
3. We conduct experiments on three public time series datasets and our results demonstrate that GenF significantly outperforms a diverse range of state-of-the-art benchmarks and classical approaches. In most cases, we find improvements of at least 10% over the benchmark methods.
To the best of our knowledge, our work the first propose using synthetic data generated by a GAN based architecture to improve long-range forecasting and the proposed cWGAN-GEP is not explored in any of the previous work.
BACKGROUND ON TIME SERIES FORECASTING
In time series forecasting, we use an observation window of historical data to predict the future value (see Fig. D. Let M denote the observation window length and N denote the prediction horizon. For those methods that contain generative models, let L denote the synthetic data window length. Let X; E R be the observation at the ieh time step, where K is the number of features. In Fig. [ the prediction horizon is N. indicating that we plan to make predictions N time steps ahead, i.e., at time step t + N (c.g.. the circled entry in Fig. [T. Next, we discuss related work in Section 2.1 and shortlist several benchmark methods for comparison in Section 2.2
2.1
RELATED WORK
Early methods which use neural networks to perform long-range forecasting include (Nguyen & Chan
2004 Goswami & Srividya 1996 AI-Saba & El-Amin 1999). For example, Nguyen & Chan (2004)
propose a group of neural networks to make predictions different time steps. Several recent works
et al. 2017 Lai et al. 2018 Qiu et al. 2020: Bouktif et al. 2018 Barssou et al. 2018 attempt improve the long-range forecasting by proposing new architectures. For example, Yu
(017)
propose a Long Short- Term Memory (LSTM) (Hochreiter & Schmidhuber, 1997) based Tensor- Train Recurrent Neural Network as a module for seguuence-t--eqqeeeee
(Sutskever al. 2014) framework, called TLSTM. Moreover, Lai et al. (2018) propose Long- and Short-term Time-series network (LSTNett) which combines L.STM with autoregressive models to minimize customized loss function inspired by support vector regression (SVR) (Cortes & Vapnik (995). Another recent work (Cheng cct al. (020)) proposes Multi-Level Construal Neural Network (MLCNN) which attempts to improve the predictive performance by fusing forecasting information of different future time. One more interesting work to mention is (Ahmed et al. 2007), which combines Gaussian processes (Rasmussen 2003) and neural networks to improve the forecasting performance. Overall, all these methods can be classified into two main classes: direct forecasting and iterative forecasting (Hamzaçebi et al. 2009 Bontempi et al. 2012).
In direct forecasting, a group of models are trained to directly make predictions at different values of N. The advantage is that all models are independent of each other and hence, models do not interfere with each other. However, its performance tends to becomes worse with N due to the lack of long-range dependencies. In iterative forecasting, the model trained make predictions for the next time step (t + 1) only. Then, the same model will be used over and over again and previous predictions are used together with the past observations to make predictions for the next time step. This process is recursively repeated N times to make predictions N time steps ahead. The previous predictions can considered as synthetic data (with synthetic data window length shorten the effective prediction horizon. However, the synthetic data generated in this supervised and recursive manner is more susceptible to error propagation, meaning that a small error the current prediction becomes larger error in subsequent predictions (Taieb al. 2012 Sorjamaa & et all 2007).
N - to
2.2 BENCHMARK METHODS
The taxonomy in Section 2.1 suggests that most existing methods can be classified either direct or iterative forecasting. This motivates us to use both direct and iterative forecasting as benchmark methods. As mentioned in Section 2.1 some classical models such LSTM, Autoregressive Integrated Moving Average (ARIMA) (Box & Pierce 1970). SVR, and Gaussian Process Regression (GPR) (Rasmussen 2003) are used to form the core of many oate-offthheart methods. Hence,
2

---

0g) Under review as a conference paper at ICLR 2021
Generative Forecasting
Training AWGAN-GEP
Training Data I
Gaussian Noise
Synthetic Data
Training Data G
Fake
Test
Data Q
Generator
Generator
X(41
Discriminator
Gaussian
cWGAN GEP
XX)
Training
Gaussian Noise
X1+1
Valid
Loss [D(X)] [D(X)
XMP,
X-Pr
Predicted
Predictor
AE x( - 1)²] + 3(Xc-1 - X(41)² X~Px
Values
concatenate at the latest time step
prune the observation of the oldest time step
Figure 2: (left) eWGAN-GEP generates synthetic data for the next time step of training data P and test data Q. The modified training data IP is used train the predictor perform predictions for the modified test data Q. (right) Training of eWGAN-GEP via training data G.
use these four classical models to implement both direct and iterative forecasting. In addition, three strong baselines described in Section 2.1 i.e., TLSTM, LSTNet, and MLCNN, also selected for comparison as they are reported to provide outstanding long-range forecasting performance in the literature (Yu al. 2017 Lai et al. 2018 Cheng al. 2020). Moreover, the source code released for these works is well organized to ensure correct implementation.
3 GENERATIVE FORECASTING
In this section, we introduce the idea GenF (see Fig. 2 and describe its three components: eWGAN-GEP based synthetic data generator, LSTM based predictor and the ITC algorithm.
3.1 IDEA OF GENERATIVE FORECASTING
To improve long-range forecasting, we develop an approach called Generative Forecasting (GenF) which consists of a cWGAN-GEP based generator, to generate synthetic data for next few time steps, and predictor, to make long-range predictions based on observed and generated data, Assuming we use past 3 observations {X,, X, X, X1-2} and make predictions at t + 10, we can express GenF as
X' X't+1 g(X,, X Xz-2,
Generate synthetic data
X' 9(X' (+1 X,
(2)
X' = =9(X' (22 X' 141, 0y)
(3)
Make long-range prediction: X = fio(X', 143, X' (+2, X'(+2, 1'241,10100 010).
(4)
where y denotes the eWGAN-GEP based generator with parameter 0g and f1o the predictor with parameter 010- Let X' X'142 and X' denote the generated synthetic data for next three time steps and X 1o denote the prediction at t + 10. Note that the value of the synthetic data window length L depends on the quality of the synthetic data and [ 3 is example of L 3. It
be seen that the generator generates synthetic data via the strategy of iterative forecasting (see I (3) and the predictor makes prediction via the strategy of direct forecasting (see 44) ). We note that GenF does not belong to either direct or iterative forecasting. The key difference is that GenF leverages synthetic data shorten the effective prediction horizon while direct forecasting does not leverage any synthetic data. Moreover, unlike iterative forecasting where the synthetic data window length is L= N 1, the synthetic data window length L of GenF is much smaller than the prediction horizon N (I = 3 and N = 10 in the example of ① 4. so that it can prevent the propagation of error an early stage. In such manner, we argue that GenF attempts to balance direct forecasting and iterative forecasting.
3.2 GENERATIVE FORECASTING: THE CWGAN-GEP BASED SYNTHETIC DATA GENERATOR
GAN (Goodfellow & all 2014) and its related works have demonstrated promising results in many generative tasks (Yi et al. 2017; Xu et al. 2017 Haidar & Rezagholizadeh 2019 Zhang et al. 2017
Dong al. 2018). The first GAN applied time series data is C-RNN-GAN (Mogren (016) which
3

---

Under review as a conference paper at ICLR 2021
takes random noise input and uses LSTM as both the generator and discriminator. One follow-up work, RCGAN (Esteban et al. (0177) makes use of the conditioning information to generate synthetic medical data. Along the way, many works have explored generating synthetic data in a diverse range
of domains (e.g. sensor data (Alzantot et al. 2017). biosignal (Haradal et al. (018)) ). We note that all
these synthetic data are generated for some purpose (Dai & 2013| Larsen et al. [013]] Dumoulin
iet al.] [2116] SSriaastaaa et al.) 2015; Yoon et al. 2019). For example, (Frid-Adar et al. | 8a) makes
use of synthetic data augmentation
improve the classification results, and (Bowles et al. 2018
mixes the synthetic data with training data to enhance the model's predictive ability. However, to the best of our knowledge, none of them have considered using the synthetic data generated by the GAN based architecture, to improve long-range forecasting.
We fill this gap and propose a conditional Wasserstein GAN with Gradient and Error Penalty (Arjovsky et al. 2017]] Mirza & Osindero, 2 Gulrajaii & et al 2017) to generator synthetic data which can be used improve long-range forecasting. The structure of cWGAN-
shown in Fig. (right). As can be seen, the training data G is used condition to direct the generation of synthetic data (Mirza & Osindero 2014). Then, the synthetic data (X(+1) will be concatenated with the training data G at the latest time step, to be evaluated by the discriminator. The network configurations of cWGAN-GEP are given in
The loss function is summarized as
E Loss = E [D(X)] - X-P, [D(X]++
+ AE K ((8)l2 - 1)²]+ + 8(x++1 - X(+1)² (5)
X~P,
8~PS
We note that the first two terms in the R.H.S (5) correspond to Wasserstein distance which improves learning performance over the Jensen- -hasenn---aonnn divergence used in the original GAN (Arjovsky et al., 2017 Gulrajani& & et all 2017). The third term is the gradient penalty, corresponding to the I-Lipschitz constraint. Unique our work, we introduce the last term which attempts to minimize the mean squared error between the synthetic data and its true value. In such a manner, we argue that the generator can generate accurate synthetic data while preserving the temporal dynamics between conditioning input and generated data by minimizing the difference between the generated data distribution Pg and real data distribution P..
3.3
GENERATIVE FORECASTING: THE LSTM BASED PREDICTOR
The LSTM unit (Hochreiter & Schmidhuber 1997) is composed of a cell which remembers values over arbitrary time intervals and three gates regulate the flow of information into and out of the cell. Such design can mitigate the issue of vanishing/exploding gradients and equips LSTM with the ability of learning and remembering over long sequence of inputs. In GenE, we use LSTM as the core of the predictor and the configuration of the predictor is given in Table (left).
3.4 GENERATIVE FORECASTING: THE INFORMATION THEORETIC CLUSTERING ALGORITHM
We note that the datasets studied in this paper contain time series data for different patients, countries and sites (described in Section 4.1.. In the following, we refer to patients, countries, sites units. We note that the ad advantage of GenF is attempting to balance direct and iterative forecasting so as to improve the long-range forecasting performance. However, the price is that we need two independent datasets: datasets G and P to train the eWGAN-GEP based generator and the LSTM based predictor, respectively. It is possible randomly split the entire training dataset at unit level into datasets G and IP, but the resulting datasets may not represent the entire training dataset well. In this subsection, we address this problem by suggesting an information theoretic clustering (ITC) algorithm.
Mutual Information (MI), denoted by T(X; Y), is an idea from information theory (Cover & Thomas 2006) that quantifies the dependency between random variables X and Y. The higher the value of the MI, the better one variable can represent the other. The ITC algorithm consists of three steps: (i) assign a score to each unit by the scoring function J(P.) = Σr, en, Pyr I(P,, Pÿ). where P. ........ } is the dataset containing all units. To estimate MI, we use the KSG estimator (Kraskov et al., (004) as all features studied are continuous variables (Gao et al., 2018 2015). (ii) divide all units into Z groups (of approximately equal size) based on the descending order of all scores, where is a tunable parameter. Therefore, units with similar scores will be grouped together. Moreover, the units within the same group tend to be highly dependent on each other. (iii) randomly sample from each subgroup: this is equivalent to selecting typical units of each subgroup. Random proportional sampling from all groups gives a new training dataset G and the unsampled units form the new training dataset P. In such manner, we can select more typical units to better train the eWGAN-GEP based generator and the LSTM based predictor in GenF.
refers to the candidate unit and D- = [PI, Pg.....
4

---

Under review as a conference paper at ICLR 2021
Generative Forecasting (GenF)
LSTNet
cWGAN-GEP
Predictor
skip-length p= 5, gradient clipping = 10, epoch = 1000, dropout 0.1, batchsize 64.
Generator
Discriminator
TLSTM
(Ss, M, K)
(Ss, (M+1), K)
(S1/S2, M. K)
learning rate (Ir) decay = 0.8, Ir= le- dropout 0.1, batchsize 64, epoch 1000.
LSTM (5)
LSTM (5)
LSTM (5)
Linear (12)
Linear (12)
LSTM (5)
ML.CNN
Linear (K)
Linear (4)
Linear (4)
span 2, stride = 1, dropout = 0.2,
Reshape (Ss. 1, K)
Linear (1)
Linear (1)
gradient clipping = 10. epoch 1000.
Table 1: (left) Network configuration of Generative Forecasting. The fourth row represents the input variables and the parameter in Linear(-) is the number of output units in the linear layer. The parameter in LSTM(()) is the hidden state size. (right) Key parameters of the three selected baselines.
4 PERFORMANCE EVALUATION
In Section
we summarize the dataset information and described the experimental setup. In Section we evaluate the performance of GenF and compare it to benchmark methods. Lastly, we conduct an ablation study to evaluate the efficiency of our framework design in Section 4.4
4.1 and 4.2
4.1 DATASET DESCRIPTION
We select three public time series datasets: MIMIC-III Vital Signs dataset (Goldberger & et all 2000), Multi-Sit Air Quality dataset (Zhang & et al 2017 and World Energy Consumption dataset (WorlBBank] 2019). The MIMIC-III Vital Signs dataset contains the first 500 patients (from subject ID 23) in MIMIC-III Clinical database. For each patient, we extract features: heart rate (bpm), respiratory rate, blood oxygen saturation (%), body temperature (2F), systolic and diastolic blood pressure (mmHg). Most (over 92%) vital signs are regularly recorded at a hourly interval over a duration of 144 hours on average. The Multi-Site Air Quality dataset includes air pollutants data from 12 monitoring sites. For each site, we extract the hourly record PMIO, SO2, NO2, Og, PM2.5 and CO. We note that all features are in units of ug/m² 3 and each site has 35,000 records average. The World Energy Consumption dataset contains data from 128 countries and each country contains three annual energy consumption indicators: electricity (kWh per capita), fossil fuel (% of total) and renewable energy (% of total) from 1971 to 2014. For each dataset, a small amount of missing values are imputed using the last historical readings. Moreover, we scale all variables to [0,1] and reshape all scaled data via a sliding observation window into three dimensional dataset D E R 8x M
4.2 EXPERIMENT SETUP
In the experiment, the dataset I is randomly split into three subsets at unit level: training dataset E R (60%), test dataset Q E R.5xMxK (20%) and validation dataset E x (20%). We note that both GenF and the benchmark methods are trained using the training dataset T, and the test dataset Q is used to evaluate the performance. To ensure fair comparison, we conduct grid search over all tunable We highlight that all methods share the same grid search range and step size. Specifically, some key parameters are tuned as follows. (i) for all LSTM based networks (including GenF), the hidden state size of LSTM is tuned from 5 to 100 with step size of 5 and number of LSTM layers or dense layers are tuned from 10 with step size of 1. (ii) for SVR, the regularization coefficient is chosen from (0.1, 1, 10). (iii for GPR, the noise level a is chosen from (1 10
parammerraettrrss..m. and possible configurations using the validation dataset.
.... 1, 10).
We now show a list of tuned parameters for predicting heart rate using the MIMIC-III Vital Signs dataset. In terms of the four classical models (described in Section 2.2 used to implement iterative forecasting and direct forecasting.
(i) SVR (kernel= = rbf) and GPR (kernel = rbf) are trained using the flatted training dataset T. (ii) ARIMA (2,0,1) is trained using the past M historical values of single feature (e.g., heart rate). (iii) for LSTM, we stack two LSTM layers (each LSTM with hidden size of 10) and two fully connected layers (with size of 10 and 1, respectively). This LSTM based neural network is directly trained using training dataset T for 1000 epochs using Adam (Kingma & Ba 2014) with learning rate of 0.001, in batches of 64. As for the three selected baselines (TLSTM, LSTNet, MLCNN), we refer to the source code released by their authors and some key parameters for those baselines are summarized in Table I (right). In terms of GenF, the training and testing
5

---

Under review as a conference paper at ICLR 2021
Algorithm 1 Generative Forecasting Algorithm
Require:
Time series dataset DD the gradient penalty coefficient A=10, the mean squared error penalty coefficient
8=15, batch size m 64, Adam Optimizer (Ir 0.001).
1: Randomly split IP at unit level: training dataset (60%), test dataset Q (20%). validation dataset (20%).
2: Apply the ITC algorithm to divide the training dataset into two subsets at unit level: training dataset G
(25%), training datasct (35%).
3: Training dataset G
trann eWGAN-GEP based generator via loss function described in
4: for = 1 to L do
5: eWGAN-GEP generate, synthetic data for the next time step of training datasct P & test datasct Q-
6: Prune the observation of the oldest time step for training dataset P & test dataset
7: Concatenate synthetic data the latest time step of training dataset P & test dataset Q (see
[
8: end for
9: Modified training dataset P -+ predictor, modified test dataset Q predictor.
10: return Mcan squared error between predicted values and real values.
procedures are summarized in Algorithm] 1 We note that both the eWGAN-GEP based synthetic data generator and the LSTM based predictor in GenF are trained using Adam (Kingma & Ba 2014) with a learning rate of 0,001, in batches of 64 and sigmoid function. Furthermore, we note that the tuned parameters and configurations are similar for other datasets. Lastly, the source code will be released to enable reproducibility at the camera-ready stage.
We compare the performance of GenF to direct and iterative forecasting in Fig. 3 All results are averaged over 10 runs and the error-bars depict the standard deviation. The Mean Squared Error (MSE) between the predicted value and true value will be used to evaluate the predictive performance. In Fig. 3 we show three variants of GenF, namely GenF-1, GenF-2, GenF-3, where the numbers represent the value of synthetic data window length L. In some plots, we only show the performance of GenF-1 and Gen This is because, in those plots, GenF-3 does not outperform GenF-1 or GenF-2,, and, hence, is not shown. Lastly, the predictive performance for the time steps that are part of the generator are not shown. For example, GenF-1 indicates that we generate synthetic data for the next time step (t + 1) and make predictions for subsequent time steps (t + 2, ...). Since the value at +++ 1 is part of the generator, the predictive performance at + 1 is not shown.
4.3 PERFORMANCE COMPARISON
In Fig- (left column), we compare the performance of GenF to iterative/direct forecasting imple- mented by the four classical models. We find that the itcrative/direct forecasting implemented by LSTM generally outperforms other implementations (c.g.. GPR and SVR). This finding agrees with
the result stated (Ma et al. 2015; Lipton et al., 2015 Gers et al. 2002: Malhotra et al. 2015). For other models, ARIMA tends to work well on the small dataset (see Fig. (e). However, on larger datasets, ARIMA tends to perform worse (e.g., Fig. (a), (g)) as it uses only single feature which may not benefit from the interaction among multiple features. Some papers (Meyler et al.) 1998, Contreras et al., 2003) also point out that ARIMA is poor at predicting turning points as essentially backward looking'. Moreover,
note that the performance of direct and iterative forecasting implemented by ARIMA are the same due to its recursive nature. Lastly, the performance of kernel based methods (GPR and SVR) generally lies in between LSTM and ARIMA (see Fig. (a). (b)). We posit this is because the time series data can be non-stationary and the optimal kernel function may change over time (Wilson& & Adams 2013 Wilson al. 2016).
In Fig. 3 (right column), we compare the performance of GenF to the three selected baselines. We note that, due the design of MLCNN (i.e., leveraging the interaction between predictions), only show its performance from + 3 onwards. Moreover, the performance of GenF in the left and right columns looks different due to the different scales, but the values are the same. We find that the performance of the selected baselines generally can outperform itcrative/direcc forecasting and are comparable to some of the GenF variants. For example, the performance of MI.CNN comparable to GenF-1 when predicting fossil fuel consumption (see Fig. 3 (f)). In most cases, the performance of GenF is better than the selected baselines. For example, in Fig. (b), the performance of MLCNN at t is 113.3 while the performance of GenF-3 at + 4 is 103.2. which is approximately 10% lower. In other plots (see Fig. 3 (a), (h)), we can observe similar or greater improvements. Lastly, we have discussed with domain experts the relevance of our long-range forecasting results (e.g., at + 4) of the two vital signs. The feedback from clinicians is that the MSE performance is acceptable for
interventional purposes for ICU patients and at at-home high risk patients.

---

Under review as a conference paper at ICLR 2021
Direct Forecasting:
LSTM
SVR
GPR
Iterative Forecasting:
LSTM
SVR
GPR
ARIMA
Selected Baselines:
LSTNet
TLSTM
MLCNN
Proposed Method:
GenF-1
GenF-2
GenF-3
(a) (b): Performance of Predicting Systolic Blood Pressure Using MIMIC-III Vital Signs Dataset
250
Iterative/Direct Forecasting
Selected Baselines
Mean Squared Error
225
Systolic Blood Pressure
Mean Squared Enr
Systolic Blood Pressure
200
175
125
150
125
(a)
(b)
100
75
75
10
11 12
3
6
9
10
11
12
Prediction Horizon (Hours)
Prediction Horizon (Hours)
(c) (d): Performance of Predicting Heart Rate
Using MIMIC-II Vital Signs Dataset
220
180
Mean Squared Error
200
Iterative/Direct Forecasting
Heart Rate
Mean Squared Error
170
Selected Baselines
Heart Rate
160
180
150
160
140
140
130
120
(c)
120
(d)
100
110
100
10
3
7
9
12
Prediction Horizon (Hours)
Prediction Horizon (Hours)
(e) (f): Performance of Predicting Fossil Fuel Consumption Using World Energy Consumption Dataset
120
Iterative/Direct Forecasting
Selected Baselines
Mean Squared Error
100
Fossil Fuel Consumption
Mean Squared Error
80
Fossil Fuel Consumption
80
60
60
40
40
e
(f)
20
20
10
12
Prediction Horizon (Years)
Prediction Horizon (Years)
(g) (h): Performance Predicting NO: Emissions Using Multi-site Air Quality Dataset
1600
1200
Mean Squared Error
1400
ttrratie/DDrret forecasting
Mean Squared Error
1000
Selected Baselines
1200
NO; Emissions
NO; Emissions
800
1000
800
600
400
(g)
(h)
200
2
3
4
5
8 9 10 11 12
2
10 12
Prediction Horizon (Hours)
Prediction Horizon (Hours)
Figure 3: Performance comparison between three variants of GenF (GenF-1, GenF-2, GenF-3) and iterative/direct forecasting ((a). (c), (e), (g)). three selected baselines ((b), (d). (f), (h)). In each subfigure, we highlight the prediction target and the benchmark method used. Each plot is averaged
over 10 runs and the error-bar is the standard derivation. Moreover, the performance of GenF in the left and right columns looks different due to the different scales, but the values are the same.
7

---

Under review as a conference paper at ICLR 2021
90
observation window
synthetic data window
M
Heart Rate
80
70
True Value (0)
GEPN-GEP (10.4)
GPR (18.6)
AWGAN-CIP (24.5)
WIGAN--FERRS (144)
ARIMA (16.3)
(15.89)
(268)
SVR (28.0)
60
0
20
21
22
23
24
25
26
27
28 time step
Figure 4: The synthetic data generated by eWGAN-GEI and variants during the experiment of predicting heart rate using MIMIC-III Vital Signs dataset. As a comparison, we also show the true value (solid line) and the synthetic data generated by the iterative forecasting strategy via the four classical models. The value in parentheses is the mean squared error with respected to the true value.
4.4 ABLATION STUDY
To demonstrate the efficiency of several key components in GenF, we conduct an ablation study. Specifically, we remove one component at a time in GenF and observe the impact on performance. First, we construct several variants of cWGAN-GEP as follows.
eWGAN-GP:
The eWGAN-GEP without the mean squared error term in the loss function (i.e.,
without the 4th term the right hand side of (5)
eWGAN-RS: The cWGAN-GEP without being trained using the ITC algorithm. Instead, the
eWGAN-GEP is trained using a randomly selected subset of the entire training dataset T.
GAN: The original GAN (Goodfellow & et al 2014) without considering the Wasserstein distance. In the experiment of predicting heart rate, evaluate the synthetic data generated by eWGAN-GEP and its three variants mentioned above. We shortlist an example of generating synthetic data for Subject ID 23 and show the past observations and the generated synthetic data in Fig. 4 Moreover, we also show the true value (solid line) and the synthetic data generated by the iterative forecasting strategy via the four classical models. The value in parentheses is the MSE relative to the true value.
For the synthetic data generated by the iterative forecasting strategy, LSTM has the smallest MSE which may explain why LSTM can outperform other classical models in long-range forecasting. Next, we compare the synthetic data generated by LSTM to eWGAN-GEP, which is essentially a combination of LSTMs (see Table (left). We find that both LSTM and eWGAN-GEP can capture the rising trend of heart rate, but the synthetic data generated by eWGAN-GEP is more stable and accurate. Specifically, the performance of L.STM and eWGAN-GES synthetic data point) are comparable. Subsequently, the synthetic data generated by LSTM tends fluctuate greatly, resulting in a larger MSE. We posit this is caused by error propagation. As for eWGAN-GEE the synthetic data is also generated recursively, but the synthetic data is more stable, leading to smaller MSE, We posit this the result of minimizing the difference data distributions and similar stabilized effect can also be observed in (Zhu et al. 2017 Frid-Adar al., 2018a).
time step = 21 (the first
When comparing eWGAN-GEP to eWGAN-GP, we find that removing the MSE term in ( causes significant drop in performance of synthetic data generation, suggesting the crucial role of the MSE term. Moreover, when comparing eWGAN-GEP to eWGAN-GEP-RS. the results demonstrate that the ITC algorithm has performance, suggesting its effectiveness in selecting typical units. As compared to the original GAN, eWGAN-GEP demonstrates superior performance in generating synthetic data well. We posit this is because the eWGAN-GEP uses Wasserstein distance as part of the loss function, leading to a more stable learning process.
5 CONCLUSION
In this paper, we develop a method called Generative Forecasting which significantly outperforms benchmark methods in long-range forecasting. We point out that the success of GenF can be attributed to two components: eWGAN-GED and the ITC algorithm, where the former can generate accurate and stable synthetic data, and the later helps to select more typical units for better training in GenF.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Nesreen Ahmed, Amir Atiya, and Neamat Gayar. combined neural network/gaussian process regression time series forecasting system for the nn3 competition. 2007.
Tawfiq Al-Saba and Ibrahim El-Amin. Artificial neural networks as applied to long-term demand forecasting. Artificial Intelligence Engineering, 13(2):189-19)) 1999.
Francisco Martinez Alvarez al. Energy time series forecasting based on pattern sequence similarity. IEEE Transactions on Knowledge and Data Engineering, 23(8): (0t):1230:: 1230-1243, 2010.
Moustafa Alzantot et al. Sensegen: A deep learning architecture for synthetic sensor data generation. In 2017 IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 188-193. IEEE, 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 214-223, 2017.
Hanich Borhan Azad, Saad Mckhilef, and Vellapa Gounder Ganapathy. Long-term wind speed fore- casting and general pattern recognition using neural networks. IEEE Transactions on Sustainable Energy, (2):544-5555 2014.
Emad Barsoum, John Kender, and Zicheng Liu. HP-GAN: Probabilistic 3d human motion prediction via GAN. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 1418-1427, 2018
Gianluca Bontempi, Souhaib Ben Taieb, and Yann-Aël Le Borgne. Machine learning strategies for time series forecasting. In European Business Intelligence Summer School, pp. 62-77. Springer, 2012.
Salah Bouktif al. Optimal deep learning LSTM model for electric load forecasting using feature selection and genetic algorithm: Comparison with machine learning approaches. Energies, 11(7): 1636, 2018
Christopher Bowles al. Gan augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv: 18/0. 10863, 2018.
George E. P. Box and David A. Pierce. Distribution of residual autocorrelations in autoregressive- integrated moving average time series models. Journal of the American Statistical Association, 65 (332):1509-1522 1970.
Chris Chatfield. Time-series forecasting. CRC press, 2000.
Jiezhu Cheng, Kaizhu Huang, and Zibin Zheng. Towards better forecasting by fusing near and distant future visions. AAAI, pp. 3593-3600, 2020.
Edward Choi and al. Using recurrent neural network models for early detection of heart failure onset. Journal of the American Medical Informatics Association, 24(2):361-370, 2016.
Javier Contreras et al. Arima models to predict next-day electricity prices. IEEE Transactions on Power Systems, 18(3):101411000, 2003.
Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):273-297, 1995.
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory, 2nd edition. John Wiley & Sons, 2006.
Andrew M. Dai and Quoe Le. Semi-supervised sequence learning. In Advances in Neural Information Processing Systems, pp. 3087, 2015.
Hao- Wen Dong al. Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of 32nd AAAI Conference on Artificial Intelligence, 2018.

---

Under review as a conference paper at ICLR 2021
Vincent Dumoulin et al. Adversarially learned inference. arXiv preprint arXi:: 2016.
Cristóbal Esteban, Stephanie L. Hyland, and Gunnar Rätsch. Real-valued (medical) time series generation with recurrent conditional GANs. arXiv preprint arXiv:/706.02633: 2017.
Maayan Frid-Adar et al. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing, 321:321-331, 2018a. 2018a.
Maayan Frid-Adar al. Synthetic data augmentation using gan for improved liver lesion classification. In IEEE International Symposium on Biomedical Imaging, pp. 289-293. IEEE, 2018b.
Shuyang Gao, Greg Ver Steeg, and Aram Galstyan. Efficient estimation of mutual information for
strongly dependent variables. In Artificial Intelligence and Statistics, pp. 277-286, 2015.
Weihao Gao, Sewoong Oh, and Pramod Viswanath. Demystifying fixed ne-reestt neighbor informa- tion estimators, IEEE Transactions Information Theory, 64(8)):5299-561,, 2018.
Felix A. Gers, Douglas Eck, and Jürgen Schmidhuber. Applying LSTM to time series predictable through time-window approaches. In Neural Nets WIRN Vietri-01, pp. 193-200. Springer, 2002.
Ary L. Goldberger and et al. Physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals. Circulation, 101 e220, 2000.
Ian Goodfellow and al. Generative adversarial nets. In Advances in Neural Information Processing 2014.
Systems, pp.
Prashant Goswami and Srividya. A novel neural network design for long range prediction of rainfall pattern. Current Science, 70(6):447-455 1996.
Clive William John Granger and Paul Newbold. Forecasting economic time series. Academic Press, 2014.
Ishaan Gulrajani and et al. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems, pp. 5767-5777, 2017.
Md Akmal Haidar and Mehdi Rezagholizadeh. Textkd-GAN: Text generation using knowledge distillation and generative adversarial networks. In Canadian Conference on Artificial Intelligence, pp. 18. Springer, 2019.
Coskun Hamzaçebi, Diyar Akay, and Fevzi Kutay. Comparison of direct and iterative artificial neural network forecast approaches in multi-periodic time series forecasting. Expert Systems with Applications, (6(2):3839-38444 2009.
Shota Haradal, Hideaki Hayashi, and Seiichi Uchida. Biosignal data augmentation based on generative adversarial networks. In 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 368-371, 2018.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 14X:v:1412.6980, 2014.
Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E, 69(6):066 118, 2004.
Guokun Lai et al. Modeling long-and short-term temporal patterns with deep neural networks. In The s International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 95-104, 2018.
Anders Boesen Lindbo Larsen et al. Autoencoding beyond pixels using learned similarity metric. arXiv preprint arXiv: 1512.0900,, 2015.
Zachary C. Lipton et al. Learning to diagnose with Istm recurrent neural networks. arXiv preprint 151 2015.
10

---

Under review as a conference paper at ICLR 2021
Gang Luo. MLBCD: machine learning tool for big clinical data. Health Information Science and Systems, 3(1):1-19, 2015.
Xiaolei Ma et al. Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transportation Research Part C: Emerging Technologies, 54:187-197, 2015.
Pankaj Malhotra et al. Long short term memory networks for anomaly detection in time series, In 23rd European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, 2015.
Massimiliano Marcellino, James H. Stock, and Mark W. Watson. A comparison of direct and iterated multistep ar methods for forecasting macroeconomic time series. Journal of econometrics, 135 (1-2):499-526, 2006.
Aidan Meyler, Geoff Kenny, and Terry Quinn. Forecasting irish inflation using arima models. 1998.
Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv: 1411. 1.744, 2014.
Olof Mogren. C-RNN-GAN: Continuous recurrent neural networks with adversarial training. arXiv preprint 161iv:1611 1.09904, 2016.
Hanh H Nguyen and Christine W Chan. Multiple neural networks for long term time series forecast. Neural Computing & Applications, 13(1):90-9) 2004.
Jiayu Qiu, Bin Wang, and Changjun Zhou. Forecasting stock prices with long-short term memory neural network based on attention mechanism. PloS one, 15(1), 2020.
Carl Edward Rasmussen. Gaussian processes machine learning. Summer School on Machine Learning, pp. 63-71. Springer, 2003.
Hoo-Chang Shin et al. Medical image synthesis for data augmentation and anonymization using generative adversarial networks. In International Workshop on Simulation and Synthesis in Medical Imaging, pp. 1-11. Springer, 2018.
Antti Sorjamaa and al. Methodology for long-term prediction of time series. Neurocomputing, 70 (16-18)2261122869..) 2007.
Nitish Srivastava, Elman Mansimov, and Ruslan Salakhudinov. Unsupervised learning of video representations using lstms. In International Conference on Machine Learning, pp. 843-852, 2015.
Ilya Sutskever, Oriol Vinyals, and Quoe Le. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112, 2014.
Souhaib Ben Taieb et al. Recursive and direct multi-step forecasting: the best of both worlds. Citeseer, 2012.
Andreas S. Weigend and Neil A. Gershenfeld. Time series prediction: forecasting the future and understanding the past. Routledge, 2018.
Andrew Wilson and Ryan Adams. Gaussian process kernels for pattern discovery and extrapolation. In International Conference on Machine Learning, pp. 1067-1075, 2013.
Andrew Gordon Wilson et al. Deep kernel learning. In Artificial Intelligence and Statistics, pp. 370-378, 2016.
WorldBank. Electricity fossil fuel energy consumption. World Development Indicator, 2019.
Zhen Xu et al. Neural response generation via gan with an approximate embedding layer. In Proceedings of International Conference Empirical Methods in Natural Language Processing, pp. 617-626, 2017.
Zili Yi et al. DualGAN: Unsupervised dual learning for saage-to-image translation. In Proceedings of the IEEE international conference computer vision, pp. 2849-2857, 2017.

---

Under review as a conference paper at ICLR 2021
Jinsung Yoon, Daniel Jarrett, and Mihaela van der Schaar. Time-series generative adversarial networks.
In Advances in Neural Information Processing Systems, pp. 5508-50085555 5518, 2019.
Rose Yu et al. Long-term forecasting using tensor-train rns. Arxiv.ArXiv Prepring:1711.00000 2017.
Shuyi Zhang and et Cautionary tales on air-quality improvement beijing. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 473(2205), 2017.
Yizhe Zhang al. Adversarial feature matching for text generation. In Proceedings of the 34th International Conference on Machine Learning, volume 70, pp. 4006-4015, 2017.
Jun-Ya Yan Zhu et al. Toward multimodal image-to-image translation. In Advances in Neural Information Processing Systems, pp. 465-476, 2017.
12


Reviews:
[{"review_id": "wp6gvEgRHdQ", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThis paper proposes cWGAN-GEP for long-range forecasting of time-series data. cWGAN-GEP is a combination of data generation of data prediction, where given some observations, GAN iteratively generates a short synthetic time-series data, and an LSTM subsequently makes a long-range prediction based on the generated synthetic data. In order to train both components, the authors use information theoretic clustering. The proposed model outperforms various baselines on the prediction task using three time-series datasets.\n\nPros:\n- The paper reads very well. The entire paper seems to be carefully written with as much details included as possible. Good presentation.\n- The proposed method outperforms various baselines for time-series prediction using three different datasets.\n\nCons:\n- It is not convincing to use a complicated GAN architecture for iterative data synthesis, when a simple LSTM could already do a decent job (as shown by figure 4). The authors should expand their ablation study by replacing the GAN module with an autoregressive LSTM/Transformer (with reasonably tuned hyperparameters) to evaluate the necessity of the complicated cWGAN-GEP. \n- It is not well motivated why a two-stage approach (data synthesis first, and prediction later) is specifically helpful for long-range forecasting. The authors could have easily used an ensemble approach by using the two components to make predictions for all future time-steps and mixing (averaging or gating, etc) the predicted outcomes, which would be way easier since this will obviate the necessity of ITC. The authors should test this approach to justify the necessity of the two-stage approach.\n- The claim that the proposed hybrid approach is helpful for long-range forecasting is not strongly justified. It seems that the performance gap between LSTM-based baseline models and the proposed methods stay rather unchanged as the prediction horizon widens. Maybe the proposed architecture is just generally a more powerful time-series predictor with many working parts.\n\nAfter the author response:\nAlthough the authors conducted additional experiments, they have only partially addressed the first comment regarding the use of LSTM instead of cWGAN-GEP by showing LSTM's comparison against cWGAN-GEP in synthetic data generation, when the question was to measure the forecasting performance, not synthetic data generation performance. Therefore increasing the score to 5.", "extra_scores": {}, "extra_text": {}}, {"review_id": "TnExDeHgBeU", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:  This paper proposes a technique for long range time-series forecasting that leverages a generative model to extend the existing time series to train a time-series model to make the final prediction at horizon N.  The authors design a GAN, exploring an enhanced loss function, and provide a methodology by which to train it.  They evaluate the combined approach on a number of different time series, while comparing the technique to both traditional forecasting approaches as well as more recent RNN/CNN-based methods.  On those datasets, the technique meets or exceeds performance measured by MSE, in some cases improving MSE by 10% or more.  \n\n=== pros ===\n\n+ Long range forecasts are indeed important, and the work appears to be a novel amalgam of recursive and direct forecasting with synthetic data generated from a GAN.  The paper is well organized and citations are extensive.\n\n+ The evaluation compares against a range of traditional and current forecasting techniques, and for those experiments illustrates performance that almost always exceeds that of other approaches. \n\n+ The ablation study indicates that the modifications to the traditional GAN architecture (extended loss function) and the use of mutual information to best train the GAN and predictive model provide benefits.   \n\n=== cons ===\n\n- The paper chooses to use MSE to compare models.  This metric has some drawbacks; it is not at the scale of the data and it amplifies changes in error.   The paper claims 10% improvements in some time series, but this is 10% WRT MSE.  MSE inflates the perceived benefit.   Using MAE or RMSE would be more appropriate, though time series challenges even eschew those metrics in favor of symmetric MAPE (sMAPE) and MASE (see M4).  This raises the possibility that the concluding statement of \"significantly outperform(ing)\" other techniques may be a bit exaggerated. \n\n- The evaluation investigates hourly and yearly time series.  It would be good to at least include a daily time series, and try to use a legitimately long-range forecasting horizon, like 30, 60, or 90 days.  \n\n- It's great to have the ablation study, but it is essentially anecdotal, as it only explores the performance for a single time series. \n\n- Model comparisons are typically performed by time series cross validation (or in the case of hyperparameter searches, nested TS CV).   This would help to remove the possiblity of leakage when training on time series that might have correlations across time across sites (such as energy consumption or air quality).  \n\n=== suggestions / questions === \n\nI enjoyed this paper and its topic.  Multi-step ahead forecasts always force the practitioner to choose between direct, recursive, or seq2seq modeling approaches.   Evaluating a new technique that combines the two is a useful exploration.   \n\nIt would be good to quantify what is meant by \"long-range\" forecasting.   Is it in steps or time?   As noted above, sometimes you have to forecast perhaps hundreds of steps into the future.  I'd probably call the TS forecasts here \"medium-range\", though this does not invalidate your results. \n\nThe motivation for ITC could be fleshed out -- it's a core contribution.   It would be good to motivate why a uniform random sampling of TS at the unit level does not lead to the best training data sets.   ITC ranks each unit time series by the sum of its MI score with all other units, and then creates Z same sized groups with similar scores.  Might explicitly say the sampling was proportional to J.   S3.4 claims that units with the same J scores tend to be dependent on each other.  What about the group with the lowest scores? \n\nIt is not clear whether you tune the hyperparameters of each direct forecasting model for each horizon.   \n\nBecause you balance direct and recursive forecasting, a discussion comparing to RECTIFY (another approach that combines the two that you cite) might be useful in the related work section.  In addition, it would be good to mention that the Yoon 2019 NeurIPS paper trained predictive models on the generated time series; it might not have been \"long range\" but it's an important aspect of that work. \n\nThere are so many variations on your approach -- it would be great to say why you didn't evaluate them.  They include using the generative model for predictions (directly and recursively), or using an interative predictor instead of a direct predictor.  Similarly, why just generate 3 steps?  \n\nS3.4 You mention that ARIMA models multi-step prediction performance is equivalent between direct and recursive.  A direct N-step ahead model would have to use lags at least N days in the past.  Other work has found recursive outperforming direct (https://arxiv.org/pdf/1108.3259.pdf).   In our own work we have found that, for AR models, it depends on the time series itself.  \n\n==== nits ==== \n\nPlease learn the difference between that and which.   Remove \"in order to.\"   \n\nAbstract \"for THE next few time steps\"\n\nS3.2P1 \"improve classification results\"\nS3.2P2 \"to generate synthetic\"\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "C27dCZknLu3", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents a framework for long-range forecasting of time series data using synthetic data. The whole pipeline contains three main components: cWGAN-GEP to predict/generate the following data based on observed ones; an LSTM predictor to forecast results based on real/generated data; and ITC to cluster data points by mutual information which forms training data for the first two components according to clustering results. The proposed framework is evaluated on three public datasets and results seem promising.\n\nPros:\n\n1. This paper is well-structured and easy-to-follow.\n2. The idea of utilizing synthetic data for improving the long-range forecasting of time series data is interesting.\n3. The proposed method is widely evaluated on three datasets and the results are promising.\n\nCons:\n\nMy main concern towards this paper is that it lacks novelty on the technical side.\n\n1. In cWGAN-GEP, adding MSE loss (mean squared error) for data generation (or augmentation) is a quite standard principle in the literature. It is hard to say this is novel.\n2. Using LSTM for long-range data prediction is also widely used given LSTNet and TLSTM.\n3. I think the main technical contribution of work is the Information Theoretic Clustering (ITC) algorithm.\n\nTherefore, the contributions of this work are over-claimed by the authors in the introduction part and its technical contribution is minor. Overall, I think the current version of this paper cannot meet the bar of ICLR.\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper tackles the problem of long term time series forecasting. One challenge in long term forecasting is that often no sufficient date may be available. This paper proposes to use GANs to generate data that can be used to improve long-range forecasts.

While reviewers agree that this paper presents an interesting idea towards tackling the problem of long term time series forecasting and appreciate the effort authors put forward in addressing their concerns and comments during the response period, the reviewers believe that in the current form paper is not ready for the publication. In particular:

1. Reviewers find that overall technical contribution of this work is limited compared to other submissions.
2. Comparison of LSTM and cWGAN-GEP in the response is only on synthetic data, which does not address reviewers concerns.




Decision: Reject