MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data
Authors: Zhibo Zhu
Ziqi Liu
Ge Jin
Zhiqiang Zhang
Lei Chen
JUN ZHOU
Jianyong Zhou
Venue: NeurIPS 2021 Poster (2021)
Keywords: Macroscopic time series forecasting
Clustering
Mixture model
Neural networks

Abstract: Time series forecasting is widely used in business intelligence, e.g., forecast stock market price, sales, and help the analysis of data trend. Most time series of interest are macroscopic time series that are aggregated from microscopic data. However, instead of directly modeling the macroscopic time series, rare literature studied the forecasting of macroscopic time series by leveraging data on the microscopic level. In this paper, we assume that the microscopic time series follow some unknown mixture probabilistic distributions. We theoretically show that as we identify the ground truth latent mixture components, the estimation of time series from each component could be improved because of lower variance, thus benefitting the estimation of macroscopic time series as well. Inspired by the power of Seq2seq and its variants on the modeling of time series data, we propose Mixture of Seq2seq (MixSeq), an end2end mixture model to cluster microscopic time series, where all the components come from a family of Seq2seq models parameterized by different parameters. Extensive experiments on both synthetic and real-world data show the superiority of our approach.

MixSeq: Connecting Macroscopic Time Series Forecasting with Microscopic Time Series Data
Zhibo Zhu"
Ziqi Liu" Ant Group
Ant Group
gavin. www.innzzbeentgruu...o com
ziqiliuGantgroup.. com
Ge Jin
Zhiqiang Zhang
Lei Chen
Ant Group
Ant Group
Ant Group
com
1 lingyao. eoagyaao.zq@aangrouppp.o com
qingli. cl@anngrrupp.cooom com
Jun Zhou
Jianyong Zhou
Ant Group
Ant Group
jun. thhoujn@@ntgroupp.ccmmm com
neil. meil.ziy@antgroup.com com
Abstract
Time series forecasting is widely used in business intelligence, e.g., forecast stock market price, sales, and help the analysis of data trend. Most time series interest macroscopic time series that aggregated from microscopic data. However, instead of directly modeling the macroscopic time series, rare literature studied the forecasting of macroscopic time series by leveraging data the microscopic level. In this paper, we assume that the microscopic time series follow some unknown mixture probabilistic distributions. We theoretically show that as identify the ground truth latent mixture components, the estimation of time series from each component could be improved because of lower variance, thus benefitting the estimation of macroscopic time series as well. Inspired by the power of Seq2seq and its variants on the modeling of time series data, we propose Mixture of Seq2seq (MixSeq), an end2end mixture model to cluster microscopic time series, where all the components come from a family of Seq2seq models parameterized by different parameters. Extensive experiments on both synthetic and real-world data show the superiority of our approach.
Introduction
Time series forecasting has proven be important to help people manage resources and make decisions [20]. For example, probabilistic forecasting of product demand and supply in retails (2], the forecasting of loans [I] in a financial institution planning to maximize the profit. Most time series interest are macroscopic time series, e.g., the sales of an online retail platform, the loans of financial institution, the number of infections caused by some pandemic diseases in a state, that are comprised microscopic time series, e.g. the sales of merchant in the online retail, the loans from customer given the financial institution, the number infections a certain region. That is, the observed macroscopic time series are just the aggregation
help people do inventory financing
of microscopic time series.
"Equal contribution. Correspnniing author.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

Although various time series forecasting models, e.g., State Space Models (SSMs) [3] Autoregres- sive (AR) models (2), or deep neural networks [5]. have been widely studied for decades, all of them study the modeling of time series without considering the connections between macroscopic time series of interest and the underlying time series on the microscopic level.
In this paper, we study the question whether the forecasting of macroscopic time series can be improved by leveraging the underlying microscopic time series, and the answer yes. Basically, though accurately modeling each microscopic time series could be challenging due to large variations, we show that by carefully clustering microscopic time series into clusters, i.e., clustered time series, and using canonical approaches to model each of clusters, finally we can achieve promising results by simply summing over the forecasting results of each cluster.
To be more specific, first, we assume that the microscopic time series are generated from a proba- bilistic mixture model [24] where there exist K components. The generation of each mieroscopic time series is by first selecting a component = from (1..., K) with a prior p(z) (a Discrete distri- bution), then generating the microscopic observation from probabilistic distribution p(æ; 22, 2) parameterized by the corresponding component 2.. We show that as we can identify the ground truth components of the mixture, and the ground truth assignment of each microscopic observation, inde- pendent modeling of time series data from each component could be improved due to lower variance, and further benefitting the estimation of macroscopic time series that are of interest. Second, inspired by recent successes of Seq2seq models [36 10 12] based on deep neural networks, e.g., variants of recurrent neural networks (RNNs) [66
convolutional neural networks (CNNs) 4 [5]. we propose Mixture of Seq2seq (MixSeq), mixture model for time series, where the components come from family of Seq2seq models parameterized by different parameters. Third, we conduct synthetic experiments to demonstrate the superiority of our approach, and extensive experiments on real-world data to show the power of our approach compared with canonical approaches.
and Transformers [99 38),
Our contributions. We summarize our contributions in two-fold. (1) We show that by transforming the original macroscopic time series via clustering. the expected variance of each clustered time series could be optimized, thus improving the accuracy and robustness for the estimation of macroscopic time series. (2) We propose MixSeq which is an end2end mixture model with each component coming from family of Seq2seq models. Our empirical results based on MixSeq show the superiority compared with canonical approaches.
2 Background
In this section, we first give formal problem definition. We then review the bases related this work, and have a discussion of related works.
Problem definition. Let us assume a macroscopic time series Titto [x1,..., Ttol and Ty E IR denotes the value of time series at time t. We aim to predict the next 7 time steps, i.c. +itttr We are interested in the following conditional distribution
to+
-1 =
II t1
p(
where I<t represents F1æ-1 in interval [1,t). To study the above problem, we assume that the macroscopic time series is comprised of microscopic time series, i.e., 222 "E Zi, where Fi, E R denotes the value of the i-th microscopic time series at time t. We aim to cluster the m
microscopic time series into K clustered time series { (x) 1 K
[1ito] ==1" where ar{²) E(ile,=avv)) Ti, given the label assignment of the i-th microscopic time series 2; E {I,...,K} This is based on our results in Section 3 that the macroscopic time series forecasting can be improved with optimal clustering. Hence, instead of directly modeling p( we study the clustering of m microscopic time series in Section 4 and model the conditional distribution of clustered time series K
(p(ala) (x) ip b)) ==1 with canonical approaches.

---

2.1 Seq2seq: encoder-decoder architectures for time series
An encoder- based neural network models the conditional distribution Eq. ① as distribution from the exponential families, e.g., Gaussian, Gamma or Binomial distributions, with sufficient statisties generated from a neural network. The encoder feeds << into neural architecture, e.g.. RNNs, CNNs or self-attentions, to generate the representation of historical time series, denoted as hy, then we use decoder to yield the result xy. After T iterations in an autoregressive style, finally generates the whole time series to be predicted.
To instantiate above Seq2seq architecture, we denote our where or E Rd, as covariates that are known priori, e.g., dates. We denote Y, = [21:--1 02:1] E ((==1)((+11) where we use || for concatenation. The encoder generates the representation h2 of ø:g via Transformer 36 19] as follows. We first transform Yú by some functions p(:), c.g., causal convolution [19] to (0) = p(Y)) E R2 (1xx))
Transformer then iterates the following self-attention layer L times:
QUOKU
1 = MMPP( (H( (tmp)) = SOFTMAX
M vl,
Vdg
(2)
Q HI ,K H(-1) (1) v
HU-"
That is, we first transform y into query, key, and value matrices, i.c., Q == Y K = YW and V = YW respectively, where Wg E Rddexd, W₁ E Rdexd,, W. E Rddxdd in each layer are learnable parameters. Then we do scaled inner product attention to yield H11) E ((((1)xdd where M a mask matrix to filter out rightward attention by setting all upper triangular elements to -c0. We denote M-P(-) as multi-layer perceptron function. Afterwards, we can generate the representation hh E Rd for Jeet via ha = ==( (F(()) where we denote v--) as a deep set function
42) that operates
(1)))). We denote the feedforward function to generate (T) as on rows of HL),, i.e. v(H L))
(LL) ()H i.e., Eq 2
Given he, the decoder generates the sufficient statisties and finally yields 2, p(x; MhPh))) from a distribution in the exponential family.
2.2 Related works
Time series forecasting has been studied for decades. We summarize works related to time series forecasting into two categories. First, many models come from the family of autoregressive integrated moving average (ARIMA) 27,) where AR indicates that the evolving variable of interest is regressed on its own lagged values, the MA indicates that the regression error is actually a linear combination of error terms, and the "I" indicates that the data values have been replaced with the difference between their values and the previous values to handle non-stationary (27). The State Space Models (SSM) [I3 aim to use state transition function to model the transfer of states and generate observations via observation function. These statistical approaches typically model time series independently, and most of them only utilize values from history but ignore covariates that are important signals for forecasting. Second, as rapid development of deep neural networks, people started studying many neural networks for the modeling of time series [5]
20 Most successful neural networks are based
on the encoder-decoder architectures (36 10 12
33
3
11
20 21), namely Seq2seq. Basically,
various Seq2seq models based on RNNs 16
22 CNNs 4 (5), and Transformers 19 38 are proposed to model the non- lineaiiyy for time series.
32
37
18
41
(self-attentions)
No matter models studied in statistics or deep neural networks, these works mainly focus on the forecasting of single or multivariate time series, but ignore the auxiliary information that the time series could be made up of microscopic data.
Time series clustering is another topic for exploratory analysis of time series. We summarize the literature into three categories, i.c., study of distance functions, generative models, and feature extraction for time series. First, Dynamic time wrapping 28). similarity metric that measures temporal dynamics (40), and specific measures for the shape (26 of time series are proposed to adapt to various time series characteristics, e.g., scaling and distortion. Typically these distance functions are mostly manually defined and cannot generalize to more general settings. Second, generative
We ignore the subscript for simplicity in condition that the context is of clarity.
3

---

model based approaches assume that the observed time series is generated by an underlying model,
such as hidden markov model 25] or mixture of ARMA 39 Third, early studies on feature extraction of time series are based on component analysis 14], and kernels, e.g., u-shapelet [43]
As the development of deep neural networks, several encoder-decoder architectures 23 are
proposed to learn better representations of time series for clustering.
However, the main purpose of works in this line is conduct exploratory analysis of time series, while their usage for time series forecasting has never been studied. That is, these works define various metries to evaluate the goodness of the clustering results, but how to learn the optimal clustering for time series forecasting remains an open question.
3 Microscopic time series under mixture model
We analyze the variance of mixture models, and further verify our results with simple toy examples.
3.1 Analyses on the variance of mixture model
In this part, we analyze the variance of probabilistic mixture models. A mixture model 24 is probabilistic model for representing the presence of subpopulations within an overall population. Mixture model typically consists of a prior that represents the probability over subpopulations, and components, each of which defines the probability distribution of the corresponding subpopulation. Formally, we can write
f(x) -w)==>m:
Pc Pe- fi(t),
where f()) denotes the mixture distribution, Pt denotes the prior over subpopulations, and fi(-) represents the distribution corresponding to the i-th component.
Proposition 1. Assuming the mixture model with probability density function (xx), and corrspond- ing components {f(x))}==
K K with constants {pi) (p)))= lie in a simplex), we have f(x)
Eedr(( condition that J() and (s(()) have first and second moments, i.e, y(t) and y(2) for f(x), and {ul (1)]
Sim1 and { 11 i=1
(2) K
for components (f((x))=m> have:
Eei ps. Var(fi) S Var(f).
(4)
We use the fact that (k) E.P.4
(t), By using Jensen's Inequality on E.p, (n
(u())²
(Expun")) we immediately yield the result. See detailed proofs supplementary.
This proposition states that, if we have limited data samples (always the truth in reality) and in case know the ground truth data generative process a priori, i.e., the exact generative process of each sample from its corresponding component, the variance on expectation conditioned on the ground truth data assignment should be larger than the variance of the mixture. Based the assumption that microscopic data are independent, the variance of the aggregation of clustered data should be least larger than the aggregation of all microscopic data, i.e., the macroscopic data. So the modeling of clustered data from separate components could possibly be more accurate and robust compared with the modeling of macroscopic data. This result motivates to forecast macroscopic time series by clustering the underlying microscopic time series. Essentially. we transform the original macroscopic time series data to clusters with lower variances using a clustering approach. then followed by any time series models to forecast each clustered time series. After that, we sum over all the results from those clusters so as to yield the forecasting of macroscopic time series. We demonstrate this result with toy examples next.
3.2 Demonstration with toy examples
We demonstrate the effectiveness of forecasting macroscopic time series by aggregating the forecast- ing results from clustered time series.

---

Table I: We run the experiments 5 times, and show the average results (SMAPE) of macro results and clustered results with ground truth clusters, Lower is better.
GP time series data
ARMA time series data
3 clusters
clusters
clusters
clusters
macro results
0.0263
0.0242
0,5870 0.3590
0.5940
clustered results
0.0210
0.0198
0.3840
Simulation setting. We generate microscopic time series from a mixture model, such as Gaussian process (GP) [3]] or ARMA g with 3 or 5 components. We generate 5 time series for each component, and yield 15 or 25 microscopic time series in total. We sum all the time series as the macroscopic time series. We get clustered time series by simply summing microscopic time series from the same component. Our purpose is to compare the performance between forecasting results directly on macroscopic time series (macro results) and sum of forecasting results of clustered time series (clustered results). We set the length of time series as 360, and use rolling window approach for training and validating model using the time series before current time point, and validate using the following 30 values). We fit the data with either GP or ARMA depending the generative model. We describe the detailed simulation parameters of mixture models in supplementary.
results in the last 120 time steps (i.e., at each time step, we train the
Simulation results. Table ! shows the results measured by symmetric mean absolute percentage eтror (SMAPE It is obvious that no matter time series generated by mixture of GP or mixure of ARMA, the clustered results are superior to macro results. In other words, if we knew the ground truth component of each microscopic time series, modeling on clustered data aggregated by time series from the same component would have better results compared with directly modeling the macroscopic time series.
MixSeq: a mixture model for time series
Based on the analysis in Section 3 we assume microscopic time series follow a mixture distribution, and propose a mixture model, MixSeq, to cluster microscopic time series.
4.1
Our model
Our model assumes that each of m microscopic time series follows a mixture probabilistic distribution. To forecast 2to+1 given Z1:to our approach is to first partition {0r,1:t]]""] into K clusters via MixSeq. Since the distribution is applicable to all microscopic time series, we ignore the subscript i and time interval 1 to for simplicity. We study the following generative probability of a e IR
pxz) = MMx, -
En(e)o(ol=)) =
= Ep(z) II p( 22)
(5)
where E {1, 2.., K} is the discrete latent variable, K is the number of components in the mixture model, p(=) ) is the prior of cluster indexed by and p(z[z) is the probability of time series x generated by the corresponding component governed by parameter d. Note that we have K parameters θ { 1 2 ... Φx) in the mixture model.
We use ConvTrans introduced in 19 as our backbone model the conditional p(z &.). To instantiate, we model the conditional p(z, F 2; 42) by first generating (0) p(Y,) via causal convolution We then generate (T) (H((0) (0) E R (txx)) given F<t- Finally we generate the representation for Zxz as h1 = (4) ( " E Rd,, where W, E Idd, xd, and as ReLU activation function. Afterwards, we decode h to form the specific distribution from an exponential family. particular, we use Gaussian distribution p(z, / 2; ..) NZ He. of). where the mean and variance can be generated by following transformations.
H = H he + bys of = log(1 + exp(w" he + b.))
(6)
'Details are in supplementary
5

---

where W/, Wo E R are parameters, and bu. bg E R are biases.
4.2 Posterior inference and learning algorithms
We aim to learn the parameter @ and efficiently infer the posterior distribution of p(=[2) a in Eq. (5) However, it is intractable to maximize the marginal likelihood p(z) after taking logarithm, i.e., log p(r), since of the involvement of logarithm of sum. To tackle this non-convex problem, resort to stochastic auto-encoding variational Bayesian algorithm (AEVB) [17]. Regarding single microscopic time series, the variational lower bound (LB) [17 on the marginal likelihood below.
Σ (2/2)) log p(x,z) x)
log p(z) = log Ep(e, 2) 2
(<((x)
(x]()) log (ogp(212)) KL (g(=|=))lp(2)) LB,
where ((2(2r) is the approximated posterior of the latent variable given time series The benefit of using AEVB that we can treat (<([z)) as an encoder modeled by neural network. Hence, we reuse the ConvTrans [9 our backbone, and model ((2/2z) as:
q(2|x) = SOFTMAX ((H()))), H(4)= = =((p(X(o)),
(8)
where we denote Y,, [x ||0:1,, E R(ox(a+1), (H()) o(W, E with parameter
W, E IR xdx is the deep set function, and Wa E Kxdd as parameters to project the encoding to K dimension. After the softmax operator, we derive the posterior distribution that lies in simplex of K dimension. Note that we use distinct p(-)'s, g(-)'s 's and (-)'s with different parameters to model g( 22)z)
and [x<e, 2) respectively. We assign each microscopic z; to cluster 2; arg max q( 2 222;)
our experiments.
Mode collapsing, We find that directly optimizing the lower bound in Eq. 7 suffers from the mode collapsing problem. That is, the encoder g(2) (())z) tends to assign all microscopic time series one cluster, and does not effectively distinguish the data expected, thus implying T(x; z) = 0 (T(-) for mutual information). In order to address the above mode collapsing problem, we add [x; 2) to the lower bound in Eq. 7 which expects that the latent variable can extract discriminative information from different time series 44 Then, we have
Ex ([(()) log p(xl))) E. (KL((<))))l(())) + ((x; =)
E, (En( in) log p(zl())) (L(()))lp())),
where q(2) =
E (<(/z;))))))))))))<()) is an average of approximated posteriors over all microscopic data.
We approximate this term by using mini-batch of m' samples, i.e., q(2) / Eres (2/2z,).
Annealing tricks. Regarding long-length time series, the reconstruction loss and KL divergence in Eq. (9) are out of proportion. In this situation, the KL divergence has few effects on the optimization objective. So we finally derive the following objective to maximize:
Eg (Eg(=)=) log p(x]=)) a (L((())llp())) - X |||||
(10)
where a is the trade-off hyperparameter. We use the following annealing strategy a max(a, x e(-fn) to dynamically adjust in the training process, where 3 is the parameter controlling the rate of descent. Meanwhile, we also involve the -norm regularizers on Seq2seq's parameters with hyperparameter À 2 0.
5 Experimental results
We conduct extensive experiments to show the advantage of MixSeq. We evaluate the clustering performance of MixSeq on synthetic data, present the results of macroscopic time series forecasting on real-world data, and analyze the sensitivity of the cluster number of MixSeq.
5.1 Synthetic datasets
To demonstrate MixSeq's capability of clustering microscopic time series that follow various proba- bilistic mixture distributions, we conduct clustering experiments on synthetic data with ground truth.

---

Table 2: Mean and standard deviation (SD. in bracket) of Rand Index (RI, the higher the better) by clustering on synthetic data generated by ARMA and DeepAR. MixSeq-infer represents that we infer the cluster of new data generated by different models after training MixSeq. On ARMA data, MixSeq and MixARMA have comparable performance; on DeepAR data, MixARMA degrades significantly which shows the effectiveness of MixSeq.
ARMA synthetic data
DeepAR synthetic data
2 clusters
3 clusters
2 clusters
3 clusters
MiXARMA MixSeq
0.9509(0.108)) 0.9082(0.0001)
0.7995(0.2734)
0. 0.7687(0.0226)
0.9915(0.0024)
0.9540(0.0974)
0.9986(0.0003)
0.8460(0.0774) 0.8460(0.0775)
MixSeq-infer
0.9929(0.0027)
0.9544(0.0975)
0.9982(0.0006)
We generate two kinds of synthetic time series by ARMA 8 and DeepAR 32 respectively. For each model, we experiment with different number of clusters (2 and 3) generated with components governed by different parameters.
Experiment setting. To generate data from ARMA, we use ARMA(2, 0) and x; = -11 222;-2+ with E (0,0027) We set parameters [0t, d2] for three components as [-0.25, 0.52]. [0.34, 0.27], and -0.5,-..75) respectively. The synthetic time series from a mixture of components are generated using the first two components. The synthetic time series from a mixture of compo- nents are generated using all 3 components. To generate data from DeepAR, we use the DeepAR model with one LSTM layer, and the hidden number of units is 16. Since it is difficult to randomly initialize the parameters of DeepAR, we train a base model on the real-world Wiki dataset [35] (discussed in section 5.2). .. To build the other two DeepAR components, we respectively add random disturbance (0, 0.01) to the parameters of the base model. For each cluster, we generate 10, 000 time series with random initialized sequences, and set the length of time series as 100.
We use l-layer causal convolution Transformer (ConvTrapy [19 as our backbone model in MixSeg. We use the following parameters unless otherwise stated. We set the number of multi-heads as kernel size as 3, the number of kernel for causal convolution d2 16, dropout rate as 0.1, the penalty weight on the fz-norm regularizer as Ie-5, and d, = d, = 16. Meanwhile, we set the prior p(=) as 1/K, 22. For the training parameters, we set the learning rate as le-4, batch size as 256 and epochs as 100. Furthermore, the a in MixSeq is annealed using the schedule a = max(5, 20e(-0.03m)), where n denotes the current epoch, and a updated in the [10, 30, 50]-th epochs. For comparison. we employ MixARMA [39, mixture of ARMA(2, 0) model optimized by EM algorithm (6. as our baseline. Both methods are evaluated using Rand Index (RI) 29 (more details in supplementary).
Experiment results. We show the clustering performance of MixSeq and MixAR on the synthetic data in Table 2 The results are given by the average of 5 trials. Regarding the synthetic data from ARMA, both MixSeq and MiXARMA perform very well. However, for the synthetic data from DeepAR, MiXARMA degrades significantly while MixSeq achieves much better performance. This suggests that MixSeq can capture the complex nonlinear characteristics of time series generated by DeepAR when MixARMA fails to do so. Furthermore, we also generate new time series by the corresponding ARMA and DeepAR models, and infer their clusters with the trained MixSeq model. The performance is comparable with the training performance, which demonstrates that MixSeq actually captures the generation mode of time series.
5.2 Real-world datasets
We further evaluate the effectiveness of our model on the macroscopic time series forecasting task. We compare MixSeq with existing clustering methods and tfate-of-the-art time series forecasting approaches on several real- real-world datasets. Specifically, for each dataset, the goal is to forecast the macroscopic time series aggregated by all microscopic data. We cluster microscopic time series into groups, and aggregate the time series in each group to form the clustered time series. Then, we train the forecasting models on the clustered time series separately, and give predictions of each clustered time series. Finally, the estimation of macroscopic time series is obtained by aggregating all the predictions of clustered time series.
7

---

Table 3: Real-world dataset summary.
# microscopic time series
length of time series
dataset
train interval
test internal
Rossmann
1115
942
20130101-20141233
20150101-20150731
M5
30490
1941
201 1011009-2016010
20160101-20160619
Wiki
309765
1827
20150701-20191231 20200101-20200630
Table 4: Comparisons the microscopic time series clustering methods for macroscopic time series forecasting combined with three network-basee forecasting methods: testing Ro.5/ gloss on three real-world datasets. Lower is better.
Macro
DTCR
MixARMA
MixSeq
DeepAR
0.1904/0.0860
0.2292/0. 1432
0.1981/0. (1981/0.1300 1300
0.1857/0.0987
Rossmann
TCN
0.1866/0.100
0.2023/0. 1633
0.1861/0.1160
0.1728/0.0997
ConyTrane
0.1861/0.0822
0.2077/0.0933
0.1866/0.0854
0.1847/0.0813
DeepAR
0.0548/0.0289
0.0787/0.062
0.0624/0.0589
0.0582/0.0444
M5
TCN
00790/0.0633
0.0847/0. 0
0.0762/0.0785 07789
0.0694/0.055
ConyTrans
0.0553/0.0260
0.0514/0.0266
0.0497/0.025
0.0460/0.0233
DeepAR
0.0958/0.096
0.1073/0. 1336
0.0974/0. 1070
0.0939/0.099
Wiki
TCN 0.0966/0.100 1064
0.1237/0. 1480
0.0963/0 1218
0.0886/0.099
Cony 0.0968/0.0589
0.1029/0.0531
0.0961/0.0594
0.0901/0.0516
We report results on three real-world datasets, including Rossmant M and Wiki 35 The Rossmann dataset consists of historical sales data of 1, 115 Rossmann stores recorded every day. Similarly, the M5 dataset consists of 30, 490 microscopic time series as the daily sales of different products ten Walmart stores in USA. The Wiki dataset contains 309, 765 microscopic time series representing the number of daily views of different Wikipedia articles. The dataset summary is shown in Table together with the setting of data splits.
setting. We summarize the clustering strategies for macroscopic time series forecasting as follows. (1) "DTCR" [21] is the deep temporal clustering representation method which integrates the temporal reconstruction, K- K-means objective and auxiliary classification task into a single Seq2seq model. (2) "MiXARMA" 39 is the mixture of ARMA model that uses ARMA to capture the characteristics of microscopic time series. (3) "MixSeq" is our model with I I-layer causal convolution Transformer [19). (4) We also report the results that directly build forecasting model the macroscopic time series without leveraging the microscopic data, named as "Macro".
Experiment
For time series forecasting, we implement five methods combined with each clustering strategy, including ARMA [8) Prophet (34), DeepAR (32), TCN 4., and ConvTrans [T9]. ARMA and Prophet give the prediction of point-wise value for time series, while DeepAR, TCN and ConvTrans are methods based on neural network for probabilistic forecasting with Gaussian distribution. We use the rolling window strategy on the test interval, and compare different methods in terms of the long-term forecasting performance for 30 days. The data of last two months in train interval are used as validation data to find the optimal model.
We do grid search for the following hyperparameters clustering and forecasting algorithms, i.e.. the number of clusters (3, 5, 7). the learning rate {0.001, 0.0001 }. the penalty weight on the 62- norm regularizers (le 5, 5e 5). and the dropout rate {0, 0.13. The model with best validation performance is applied for obtaining the results on test interval. Meanwhile, we set batch size as 1 128, and the number training epochs as 300 for Rossmann, 50 for M5 and 20 for Wiki. For DTCR, we use the same setting (21]
Regarding time series forecasting models, we apply the default setting ARMA and Prophet provided by the Python packages. The architectures of DeepAR, TCN and Conv Trans are as follows.
8

---

Table 5: Comparisons on the microscopic time series clustering methods for macroscopic time series forecasting combined with five forecasting methods: testing SMAPE on three weal-world datasets.
Macro
DTCR
MiXARMA
MixSeq
ARMA
0.2739(0.0002)
0.2736(0.0013) 0.2735(0.0106)
0.2733(0.0012)
Prophet
(09007)))0007)
0.1743(0.0037) 0.17380.0013))
0.1743(0.0026)
Rossmann
DeepAR
0.1026(0.0081)
0.1626(0.011))
0.1143(0.0083
0.0975(0.0013)
TCN
0.1353(0.0254) 0.1055(0.0155)
0.1427(0.0180)
0.1027(0.0075)
ConyTrans
0.1028(0.0091)
0.1731(0.0225)
0.1022(0.0041)
(.0961().0019)
ARMA
0.0540(0.0001)
0.0544(0.0018)
0.0541 (0.0003)
0.0543(0.000))
Prophet
0.027 (0.200().0003)
0.0271(0.0003)
0.0269 0.002)
0.0267 (0.0002)
M5
DeepAR
0.0278(0.0034)
0.0410(0.0046
0.0319 (00063)
0.0298(0. 0029)
TCN
0.0412(0.0075
0.0447(0.0044)
0.0395(0.0094)
0.0358(0.0014)
Conv Trans
0.0274(0.0048)
0.0253(0.0020)
(0.05))
0.0227
ARMA
0.0362
0.0363
0.0364(0.0005
0.0362 0.002
Prophet
0.0413
0.0423(0.000$
0.0434(0.0003)
0.0420(0.000
Wiki
DeepAR
0.048 (0.0008)
0.0552(0.0015)
0.0489(0.0006)
0.0470(0.0002
TCN
0.0494(0.0076)
0.0654(0.0022)
0.0491(0.0015)
0.0446(0.0023)
Conv'Trans
0.0471 (0.029)
0.0497(0.0012)
0.0466(0.0001)
0.0440(0.001))
The number of layers and hidden units are and 16 for DeepAR. The number of multi-heads and kernel size are 2 and 3 for ConnTr The kernel size is 3 for TCN with dilations in [1, 2, 4, 8). We also set batch size as 128 and the number of epochs as 500 for all forecasting methods.
Experiment results. Following 19 30 351,
we evaluate the experimental methods using SMAPE and p-quantile loss 7 with p E (0, 1). The SMAPE results of all combination of clustering and
forecasting methods are given in Table 5 Table 4 shows the Ro.5/ Ro.g-loss for DeepAR, TCN and ConyTrans which give probabilistic forecasts. results are run in 5 trials. The best performance is highlighted by bold character. We observe that MixSeq is superior to other three methods, suggesting that clustering microscopic time series by our model is able improve the estimation of macroscopic time series. Meanwhile, Macro and MiXARMA have comparable performance and are better than DTCR, which further demonstrates the effectiveness of our method, i.e., only proper clustering methods are conductive to macroscopic time series forecasting.
5.3 Sensitivity analysis of cluster number
The cluster number K is a critical hyperparameter of MixSeq. To analyze its effect the forecasting of macroscopic time series, we conduct experiments on both synthetic data and real-world data. The results state the importance of setting proper number of clusters. We suggest do binary search on this critical hyperparameter. Details are as follows.
Synthetic data. Following the experimental setting in section 5.1 we generate 10, 000 microscopic time series from 3 different parameterized ARMA respectively. That is, the ground truth number of clusters is 3 and there are 30, 000 microscopic samples in total. The aggregation of all samples is the macroscopic time series which is the forecasting target of interest. Then, we compare the forecasting performance between the method that directly forecasts macroscopic time series (denoted as Macro) and our method with different cluster numbers (including 2, 3, 5, denoted as MixSeq_2, MixSeq_3 and MixSeq_5 respectively). We fix the forecasting method ARMA, and apply the rolling window approach for T+ 10 forecasting in the last 40 time steps. The average SMAPE of 5 trials 0.807, 0.774. 0.731 and 0.656 for Macro, MixSeq_2 2, MixSeq_3 3 and MixSeq respectively. MixSeq_3 being set with ground truth cluster number shows the best performance, while MixSeq_ 2 and MixSeq_5 would degenerate though still better than Macro. This result shows the importance of setting proper number of clusters.
Real-world data. We do the evaluation three real-world datasets by varying the cluster number K of MixSeq while maintaining the other parameters fixed. For Rossmann and M5 datasets, we set the
Detailed definition is in supplementary.

---

p 0.5 Rossmann
0.5 M5
0.5 Wkk
0.210
0.058
0.096
0.205
p-quantle loss
0.054
0.052
0.195
0.050
0.092
0.190
0.185
0.046
2
2
6
5
10
15
p 0.9 Rossmann
p = 0,9 M5
p= 0.9 WKI
0.094
0.092
0.0260
0.060
D-quantile loss
0.090
0.0255
0.058
0.0250
0.056
0.086
0.0245
0.082
0.0240
K
K
Figure The macroscopic time series forecasting performance based on MixSeq with different cluster number K on three real-world datasets. The time series forecasting method is fixed as causal convolution Transformer. Top three figures show the Ro.s-loss and bottom three figures show the Ro.s-loss.
cluster number K E (3,5, 7). while we explore the cluster number K (5, 10, 15) on Wiki dataset. The architecture and training parameters MixSeq are same section 5.2 except that we set dropout rate 0.1, the penalty weight on the f2-norm regularizer as 5e-5, and the learning rate as le-4. Mcanwhile, we also fix the time series forecasting method as causal convolution Transformer (ConvTrans)
Figure T reports the macroscopic time series forecasting performance (testing on Ro.s and Rg.9 loss) based on MixSeq with different cluster number K on three real-world datasets. The horizontal dashed lines the results with K that directly building ConvTrans model on the macroscopic time series without leveraging the microscopic data (named as "Macro" in section 5.2). It is obvious that each dataset has its own suitable number of clusters, and our method relatively sensitive to K, especially on the dataset with less microscopic time series, such as Rossmann. Similar to the accurately modeling each microscopic time series, the larger cluster number K of MixSeq also brings large variance to macroscopic time series forecasting, which degrades the performance of our method.
Conclusion
In this paper, we study the problem that whether macroscopic time series forecasting can be improved by leveraging mieroscopic time series. Under mild assumption of mixture models, we show that appropriately clustering microscopic time series into groups is conductive to the forecasting of macroscopic time series. We propose MixSeq to cluster microscopic time series, where all the components come from a family of Seq2seq models parameterized with different parameters. We also propose an efficient stochastic auto-encoding variational Bayesian algorithm for the posterior inference and learning for MixSeq. Our experiments on both synthetic and real-world data suggest that MixSeq can capture the characteristics of time series different groups and improve the forecasting performance of macroscopic time series.
10

---

Acknowledgments and Disclosure of Funding
This work is supported by Ant Group.
References
[1] Abu-Mostafa and A. Atiya. Introduction financial forecasting. Applied intelligence, 6(3): 205-213, 1996.
[2] D. Asteriou and S. G. Hall, Arima models and box-jenkins methodology. Applied Econometrics, 2(2): 265-286, 2011.
[3] D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (ICLR), 2015.
[4] S. Bai, J. Z. Kolter, and V. Koltun, An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint 1rXXiv:803.011 2018.
[5] K. Benidis, S. S. Rangapuram, V. Flunkert, B. Wang, D. Maddix, C. Turkmen, J. Gasthaus, M. Bohlke- Schneider, D. Salinas, L. Stella, et al. Neural forecasting: Introduction and literature overview. arXiv preprint arXiv:2004.102444 2020.
[6] D. M. Blei, A. Kucukelbir, and J. D. McAuliffe. Variational inference: A review for statisticians. Journal of the American stotistical Association, 112(518):859 877, 2017.
[7] G. E. Box and G. M. Jenkins. Some recent advances in forecasting and control. Journal of the Royal Statistical Society: Series C (Applied Statistics), 1 17(2)91-109, 1968.
[8] G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung. Time series analysis: forecasting and control. John Wiley & Sons, 2015.
[9] C. Chen, Z. Liu, J. Zhou, X. Y. Qi, Yia,,, and X. Zhong. How much can retailer sell? sales forecasting tmall. In Pacific-Asia Conference Knowledge Discovery and Data Mining, pages 204-216. Springer, 2019.
[10] K. Cho, B. Van Merrienhoer, D. Bahdanau, and Y. Bengio. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint 1rXiv: 2014.
[11] F. Dama and C. Sinoquet. Analysis and modeling forecast in time series: systematic review, arXiv preprint arXi:22104.0016 2021.
[12] S. Du, Li, S.-J. Horng. Time series forecasting using sequence-to-see to-sequence deep learning framework. In 20/8 9th International Symposium on Parallel Architectures, Algorithms and Programming (PAAP), pages 171-176. IEEE, 2018
[13] J. Durbin and S. J. Koopman. Time series analysis by state space methods. Oxford university press, 2012.
[14] C. Guo, Jia, and N. Zhang, Time series clustering based on ica for stock data analysis. In 2008 4th International Conference on Wireless Communications, Networking and Mobile Computing, pages 1-4. IEEE, 2008.
[15] H. Hao, Y. Wang. Y. Xia. J. Zhao, and F. Shen. Temporal convolutional attention- network for sequence modeling. arXiv preprint arXiv:2002.1 1203,0 2020.
[16] H. Hewamalage, C. Bergmeir, and K. Bandara. Recurrent neural networks for time series forecasting: Current status and future directions. International Journal of Forecasting, 37(1):388-427, 2021.
[17] D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 14, 2013.
[18] G. Lai, W.- Chang, Y. Yang, and H. Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pages 95-104, 2018.
[19] S. Li, X. Jin, Y. Xuan, X. Zhou, W. Chen, Y-X. Wang, and X. Yan. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting. arXiv preprint arXiv: 1907.0023.. 2019.
[20] B. Lim and S. Zohren. Time series forecasting with deep learning: A survey. arXiv preprint STti:2004.134000 2020.

---

[21] Q. Ma. J. Zheng, S. Li, and G. W. Cottrell. Learning representations for time series clustering. Advances in neural information processing systems, 32:3781-3791, 2019.
[22] D. C. Maddix, Y. Wang, and A. Smola. Deep factors with gaussian processes for forecasting. arXiv 2018.
preprint
[23] N. Madiraju, M. Sadat, Fisher, and Karimabadi. Deep temporal clustering: Fully unsupervised learning of time-domain features. arXiv preprint Sati::/802.000300 2018.
[24] G. J. McLachlan and K. E. Basford. Mixture models: Inference and applications clustering, volume 38. Dekker New York, 1988.
[25] T. Oates, L. Firoiu, and P. R. Cohen. Clustering time series with hidden markov models and dynamic time warping. In Proceedingy the IJCAI-99 workshop neural, symbulic and reinforcement learning methods for sequence learning, pages 17-21. Citeseer, 1999.
[26] J. Paparrizos and L. Gravano. k-shape: Efficient and accurate clustering of time series. In Proceedings of the 2015 ACM SIGMOD International Conference Manugement of Data, pages 1855-1870, 2015.
[27] J. Pemberton. Non-linear and non-stationary time series analysis., 1990.
[28] F. Petitjean, A. Ketterlin, and P. Gançarski. global averaging method for dynamic time warping, with applications to clustering. Pattern recognition, 44(3):67--693, 2011.
[29] W. M. Rand. Objective criteria for the evaluation of clustering methods. Journal of the American Staristical association, 66(336):846 850, 1971.
[30] S. S. Rangapuram, M. W. Seeger, J. Gasthaus, L. Stella, Y. Wang, and T. Januschowski. Deep state space models for time series forecasting. Advances in neural information processing systems, 31:7785-7794, 2018.
[31] S. Roberts, M. Osborne, M. Ebden, S. Reece, N. Gibson, and S. Aigrain. Gaussian processes for time-series modelling. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, (71(1)84) 10550, 2013.
[32] D. Salinas, V. Flunkert, J. Gasthaus, and T. Januschowski. Deepar: Probabilistic forecasting with autore- gressive recurrent networks. International Journal of Forecasting, 16633):181-110011 181-1191, 2020. 2020.
[33] Sutskever, O. Vinyals, and Q V. Lc. Sequence to sequence learning with neural networks. Advances in neural information processing systems, 2014.
[34] S. J. Taylor and B. Letham. Forccasting at scale. The American Staristician, 72(1):37-45, 2018.
[35] A. Tran, A. Mathews, C. S. Ong, and L. Xie. Radilow: A recurrent, aggregated, and decomposable model for networks of time series. arXiv preprint arXiv:2102.07289 2021.
[36] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need, arXiv preprint arXiv:1706.037: 03762, 2017.
[37] R. Wen, K. Torkkola, B. Narayanaswamy, and D. Madeka, A multi-horizon quantile recurrent forecaster. arXiv preprint 1i1.:171..100: 2017.
[38] N. Wu, B. Green, X. Ben, and O' O'Banion. Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprimt urXiv:2001.083// 2020.
[39] Y. Xiong and D.-Y. Yeung. Time series clustering with arma mixtures. Pattern Recognition, 37(8): 2004.
[40] J. Yang and J. Leskovec. Patterns of temporal variation in online media. In Proceedings of the fourth ACM international conference Web search and data mining, pages 177-186, 2011.
[41] R. Yu, S. Zheng, A. Anandkumar, and Yue. Long-term forecasting using tensor-train rnss. Arxiv, 2017.
[42] M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. Salakhutdinov, and A. Smola. Deep sets. arXiv preprint 1 14, 2017.
[43] J. 7akaria, A. Mueen, and F. Keogh, Clustering time series using unsuperviseedssapellss. Tn 2012 IEEE 12th International Conference on Data Mining, pages 785-794. IEFF, 2012.
[44] T. Zhao, K. Lec, and M. Eskenazi. Unsupervised discrete sentence representation learning for interpretable neural dialog generation. arXiv preprint
2018
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section 5
(c) Did you discuss any potential negative societal impacts of your work? [Yes] See
supplementary.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section
(b) Did you include complete proofs of all theoretical results? [Yes] See supplementary.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or URL)? [Yes] See Section 5 and supplementary.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Section (ectionnSaand supplementary.
(c) Did you report error bars (e.g., with respect the random seed after running ex- periments multiple times)? [Yes] See all the tables. We report mean error and its std.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes] See supplementary.
4. If you using existing assets (e.g., code, data, models) or curating/releasing assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any assets either in the supplemental material as URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you're 're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "wChpqjBVyYm", "reviewer": "Reviewer_pibM", "summary": "Authors model time series as a weighted mixture of a set of time series for forecasting. Proposed a model for Mixture of seq2seq model. Number of components depends on correct clustering of micro time series. The paper also  propose an efficient stochastic auto-encoding variational Bayesian algorithm for the posterior inference and learning for MixSeq\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The idea of using a mixture of models in time series prediction is already there [30, 39], and Proposition-1 is about any general data point and function which is known. The first contribution seems incremental. The result shows accuracy increases monotonically with the splitting of the time-series into components. There is no result or discussion then how many components are sufficient or it is really monotonic. Then how do we choose the number of components?\n The second contribution of building up the mixture model of seq2seq is new and the learning process is original. However, the use of seq2seq over RNN is not very clear. \n\nThe Significance  of the task of forecasting time series data is well known. The paper could not demonstrate the advantage of the proposed method. Theoretical result is not presented regarding this and more over Empirical evaluation is not significant. \n\nI have following question regarding Empirical set up  and claims\nVarious tables report various metrics without any proper discussion. It will be better to have consistency over comparing method and performance metric. \nSensitivity analysis on hyperparameters is not presented. It seems the author has used validation to fix hyperparameters.  But how sensitive is model performance to the number of cluster assumptions.\nNumbers reported in table 5 are very much comparable [{arma: 0.0269(0.0002) seq:0.0267(0.0002)};{0.2736(0.0013) 0.2733(0.0012)}]. Is there any statistical significance? \n\nPaper is not easy to read. Did not have discussion of state of art mixture models for time series forecasting.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Many real world data need to forecast which are important for damage management. Author can discuss if the proposed forecasting method can handle forecasting such events like extreme rain fall. Wrong forecasting may lead to lots of damage of human life and properties.", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": ["Inadequate Data and Algorithm Evaluation"], "time_spent_reviewing": "8"}, "extra_text": {}}, {"review_id": "bvhKjIPXCzF", "reviewer": "Reviewer_af7g", "summary": "This paper develops methodology for improving time series forecasting by modeling finer-scale structure in the time series; namely, the underlying \"microscopic\" time series of which the predicted \"macroscopic\" time series is aggregated. The approach takes advantage of the concept that in many cases the time series of interest are in fact macroscopic aggregates of smaller-scale structure and time series. The work develops a method that models the micro time series as a mixture of latent components and then uses a neutral network based approach to model and parameterize the mixture. Benchmarkins are done with simulated and real data, showing promising performance.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The originality relates to bringing together the underexplored idea of aggregating information across microscopic time series, and a well-motivated modeling approach. The methodology appears justified and sound and the reporting is clear. The main significance of this work is in opening up a new line of analysis for micro time series aggregation based on neural & autoencoder models, for improving time series forecasts.\n\nStrengths:\n+ Thorough analysis\n+ Pragmatic idea\n+ Clear performance improvements\n\nWeaknesses:\n- Not clear if all topics in the checklist have been properly addressed (several \"No\" questions in the form)\n- More information / examples on common settings where the assumptions break and alternative models are better would be useful\n- More information on running times would be useful\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "According to the checklist, the authors have not addressed the limitations and societal impacts.\n\nSuggestion for improvement would be to go through the checklist and address the required points in the discussion. Further evaluation is possible after this.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "PDWay9LP0cX", "reviewer": "Reviewer_viNa", "summary": "This paper tackles the problem of predicting a so called macroscopic time series that is the aggregation of multiple microscopic time series. For example, global sales of a chain can be the sum of sales in individual locations. The authors propose to predict the macroscopic time series by building a mixture model on microscopic time series. They provide a theoretical and empirical justification of this approach. They also compare their approach to baselines on synthetic and 3 real-world data sets. For the latter data sets, performance on microscopic time series is also evaluated. The results suggest the proposed approach is better than the competing baselines.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Originality**\nI think the originality of the paper is good.\n- The related work section is informative, well written and broad.\n- I am not aware of any work that specifically studies time series forecasting from microscopic time series. I think it is an interesting topic, as it happens frequently.\n- While mixture models are not new and the approach proposed here does not bring much novelty, I believe its application to the problem is nevertheless interesting.\t\n\n**Quality**\nI think the quality of the paper is average.\n- It is quite possible I am missing something, but I cannot get my head around the theoretical justification for the approach (proposition 1). I would really like the authors to address this and/or correct me in the rebuttal. It states that $\\sum_i p_i Var(f_i) \\leq Var(f)$, where $f_i$ denotes the distribution over a cluster and the mixture distribution $f(x)=\\sum_i p_i f_i(x)$. Based on this, the proposed approach is motivated by the fact that modeling individual clusters could be more accurate than modeling the mixture directly. The demonstration with toy examples is very nice though.\n   - My first question is about the demonstration. I agree that the variance of the mixture is higher than the weighted sum of individual variances. However, I am not sure that the moment of any order (k) of the mixture is the weighted sum of the corresponding cluster moments: $\\mu^{(k)} = \\sum_i p_i \\mu_i^{(k)}$   (equation 2 in the appendix). \n   - My second issue is that the goal of the paper is not to model the mixture itself, but an aggregation (the macroscopic time series) of multiple realization of the mixture (the microscopic time series). In other words, the variance of interest is not the variance of the mixture, but the variance of a sum of random variables. Hence I do not see the relevance of the theoretical justification.\n- The experiments on the other hand are very nice. The baselines seem to make sense, there are three real world and relevant data sets. The proposed approach outperforms the baselines most of the time. They also show the strength of the approach by considering variations such as only modelling the macroscopic time series. I also like that both the macroscopic and the microscopic time series forecasting are considered on the real world data sets as both are likely to be useful in practice. I think it is not necessary for the paper to be good, but it would have been nice to have some exploration of the clusters learned by the model. \n- Chosing the number of clusters is not discussed. This is not a big problem as anybody who has worked on mixture model will have faced this problem, but some pointers could be nice for readers not familiar with the topic.\n\n**Clarity**\nI think the clarity of the paper is not good.\n- The paper is well organized and easy to follow at a high level. \n- The experiments are well described. I think they could be reproduced easily.\n- However, the paper should be carefully proofread for spelling and grammatical mistakes. There are too many left and this significantly impacts readability. In my opinion in its current state the paper is not ready to be published. This is my main concern with this paper.\n\n**Significance**\nI think significance is fine.\n- The addressed problem is very specific, but it is encountered in practice.\n- The proposed approach beats the baselines very often.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an approach for forecasting a single "macroscopic" time series which is the sum of several "microscopic" time series by forecasting the microscopic time series using a mixture of transformer-based seq2seq models and finally combining the forecasts. 

The reviewers agree that the paper proposes a pragmatic solution to a relevant problem, and demonstrates clear performance improvements in doing so. While the building blocks (mixture models trained with variational inference, ConvTrans time series model) are well know, the combination is novel and well-chosen to address the task at hand. The reviewers highlight the convincing empirical evaluation against sensible baselines, where significant performance gains were demonstrated. Initial reviewer concerns around novelty and clarity were alleviated during the discussion period, leading me to recommend acceptance.
Decision: Accept (Poster)