Learning Latent Seasonal-Trend Representations for Time Series Forecasting
Abstract
Forecasting complex time series is ubiquitous and vital in a range of applications but challenging.
Recent advances endeavor to achieve progress by incorporating various deep learning techniques (e.g., RNN and Transformer) into sequential models.
However, clear patterns are still hard to extract since time series are often composed of several intricately entangled components.
Motivated by the success of disentangled variational autoencoder in computer vision and classical time series decomposition, we plan to infer a couple of representations that depict seasonal and trend components of time series.
To achieve this goal, we propose
LaST, which, based on variational inference, aims to disentangle the seasonal-trend representations in the latent space.
Furthermore, LaST supervises and disassociates representations from the perspectives of themselves and input reconstruction, and introduces a series of auxiliary objectives.
Extensive experiments prove that
LaST achieves state-of-the-art performance on time series forecasting task against the most advanced representation learning and end-to-end forecasting models.
For reproducibility, our implementation is publicly available on Github 1 .
Introduction
Time series forecasting plays a significant role in plethora of modern applications, ranging from climate analysis [1], energy production [2], traffic flows [3] to financial markets and various industrial systems [4].
The ubiquity and importance of time series data have recently attracted researcher efforts resulting in a myriad of deep learning forecasting models [5, 6] ameliorating the time series forecasting.
Based on advanced techniques such as RNN and Transformer [7]), these methods usually learn latent representations to epitomize every instant of the signals and then derive forecasting results by a predictor, achieving great progress on forecasting tasks.
However, these models have difficulties to extract exact/clear information related to temporal patterns (e.g., seasonality, trend, and level), especially in supervised end-to-end architecture without any constraint on representations [8].
As a consequence, efforts have been made to apply the variational inference into time series modeling [9, 10, 11], where improved guidance for latent representations with probabilistic form has been proved beneficial to downstream time series tasks [12].
However, when various intricately co-evolving constituents exist a time series data, analyzing with a single representation will result in superficial variable and models’ non-reusablity and lack of interpretability, due to the highly entangled nature of neural networks [13, 14].
Thus, while providing efficiency and effectiveness, existing approaches with a single high-dimensional representation sacrifice the information utilization and explainability, which may further lead to overfitting and degenerated performance.
To address the above limitations and seek a new disentangled time series learning framework, we leverage the ideas from the decomposition strategy [15, 16] to split time series data into several components, each of which captures an underlying pattern category.
The decomposition assists the analysis process and reveals underlying insights more consistently with human intuition.
This insight motivates us to produce a couple of latent representations that respond to different time series characteristics (in our case the seasonal and trend), from which we predict the results by formulating sequence as the sum of these characteristics.
The representations should be as independent as possible to avoid a model prone to feature entanglement, while also have sufficient information to the input sequence.
Towards that, we propose a novel framework LaST to learn the La tent S easonal- T rend representations for time series forecasting.
LaST exploits an encoder-decoder architecture and follows variational inference theory [17] to learn a couple of disentangled latent representations that describes seasonal and trend of time series.
To achieve this goal, LaST enforces the representations into disentanglement under two constraints: (1) from the input reconstruction, we dissect intrinsic seasonal-trend patterns which can be readily obtained from raw time series and off-the-shelf measurement methods, and accordingly design a series of auxiliary objectives; (2) from representations themselves, we minimize the mutual information (MI) between seasonal and trend representations on the premise that grantee the consistency between input data and each of them.
Our main contributions are threefold:
• We start with the variational inference and information theory to design the seasonal-trend representations learning and disentanglement mechanisms, and practically demonstrate their effectiveness and superiority (over the existing baselines) on time series forecasting task.
• We propose LaST, a novel latent seasonal-trend representations learning framework, which encodes input as disentangled seasonal-trend representations and provides a practicable approach that reconstructs seasonal and trend separately to avoid chaos.
• We introduce MI terms as a penalty and present a novel tractable lower bound and an upper bound for their optimizations.
The lower bound ameliorates the biased gradient issue in prevalent MINE approach and ensures informative representations.
The upper bound provides the
feasibility to further reduce the overlapping of seasonal-trend representations.
Related work
Most of the deep learning methods for time series forecasting are designed as an end-to-end architecture.
Various basic techniques (e.g., residual structure [18, 19], autoregressive network [20, 21], and convolutions [22, 23]) are exploited to produce expressive non-linear hidden states and embeddings that reflect the temporal dependencies and patterns.
There is also a body of works that apply
Transformer [7] structure into time series forecasting tasks [24, 25, 26, 6], aiming to discover the relationships across the sequence and focus on the important time points.
Deep learning methods have achieved superior performance in comparison to the classical algorithms such as ARIMA [27] and VAR [28], and have become prevalent in multiple applications.
Learning flexible representations has been demonstrated to be beneficial for downstream tasks by numerous researches [12, 29].
In time series representations domain, early methods, employing the variational inference, jointly train an encoder and corresponding decoder that reconstructs raw signals to learn approximate latent representations [10, 30, 31].
Recent efforts have improve these variational methods [32, 33] by establishing more complex and flexible distributions using techniques such as copula [32] and normalizing flow [34, 35].
Another group of works exploited the burgeoning contrastive learning to obtain invariant representations from augmented time series [36, 37, 38], which avoids the reconstruction process and improves representations without additional supervision.
Time series decomposition [15, 16] is a classical method that splits complex time series into several components to obtain the temporal patterns and interpretability.
Recent works have applied machine learning and deep learning approaches [39, 40, 41] to robustly and efficiently achieve the decomposition on large-scale datasets.
There are also research results which tackle forecasting with the assistance of decomposition.
For example, Autoformer [26] decomposes time series into seasonal-trend parts by average pooling and introduces an autocorrelation mechanism to empower
Transformer [7] for better relations discovery.
CoST [38] encodes signals into seasonal and trend representations in frequency and temporal domain, respectively, and introduces contrastive learning to supervise their learning.
Different from our work, these methods exploit simple average pooling decomposition mechanism which may provide incompatible periodical assumptions, or intuitively disentangle the representations by processing in different domains.
Meanwhile, LaST adaptively epitomizes the seasonality and trend by disentangled representations and boosts their disassociation from a probabilistic perspective in the latent space.
Latent seasonal-trend representations learning framework
We now formalize the problem definition and introduce the proposed LaST framework.
We note that in LaST we use seasonal and trend characteristics for disentanglement learning but our framework can be easily extended to adapt situations that have more than two components to dissociate.
Problem definition.
Consider a time series dataset D consisting of N i.i.d.
sequences denoted as X ( i ) 1: T = { x ( i ) 1 , x ( i ) 2 , · · · , x ( i ) t , · · · , x ( i ) T } , where i ∈ { 1 , 2 , . . .
, N } , and each x ( i ) t is univariate or multivariate value representing the current observation(s) at time instant t (e.g., price and rainfall).
We aim to derive a model that outputs expressive representations Z 1: T , suitable for predicting future sequence(s) Y =
ˆ X T +1: T
+ τ .
Hereafter, when there is no ambiguity we omit the superscripts and the subscript 1: T .
A model that infers the likelihood between observation X and future Y with latent representation Z can be formulated as follows:
[Equation]
From the perspective of variational inference (cf.
[17]), the likelihood P ( X | Z ) is calculated by a posterior distribution Q ϕ ( Z | X ) and maximized by the following evidence lower bound (ELBO
[Equation]
where Θ is composed of ψ , ϕ , and θ denotes learned parameters
[Equation]
[Figure]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
+gEeegS1dEtaqAmokihZ/SK3hxwXpx352PWuuQUMwfoD5zPH6iCkYI=</latexit>
˜
Z t <latexit sha1_base64="blo2+iWLmYnB0p/MRZ06BwhpEnw=">AAAB6HicbVDLTgJBEOzFF+IL9ehlIjHxRHbVqEeiF4+QyMPAhswODYzMzm5mZk3Ihi/w4kFjvPpJ3vwbB9iDgpV0UqnqTndXEAuujet+O7mV1bX1jfxmYWt7Z3evuH/Q0FGiGNZZJCLVCqhGwSXWDTcCW7FCGgYCm8Hoduo3n1BpHsl7M47RD+lA8j5n1Fip9tAtltyyOwNZJl5GSpCh2i1+dXoRS0KUhgmqddtzY+OnVBnOBE4KnURjTNmIDrBtqaQhaj+dHTohJ1bpkX6kbElDZurviZSGWo/DwHaG1Az1ojcV//Paielf+ymXcWJQsvmifiKIicj0a9LjCpkRY0soU9zeStiQKsqMzaZgQ/AWX14mjbOyd1k+r12UKjdZHHk4gmM4BQ+uoAJ3UIU6MEB4hld4cx6dF+fd+Zi35pxs5hD+wPn8AbnZjOU=</latexit> Y
[Figure]
However, as pointed out in Sec.
1, this faces and entanglement problem and cannot clearly extract complicated temporal patterns.
To ameliorate this limitation, we incorporate the decomposition strategy into our LaST that learns a couple of disentangled representations to depict seasonal and trend dynamics.
Specifically, we formulate the temporal signals X and Y as the sum of seasonal and trend components, i.e., X =
X s + X t .
Accordingly, the latent representation Z is factorized into Z s and Z t , assumed to be independent of each other – i.e., P ( Z )
=
P ( Z s ) P ( Z t ) .
Figureillustrates the two parts of the LaST framework: (a) representations learning, producing disentangles seasonal-trend representations (separate reconstructions and MI constraints); and (b) prediction based on learned representations.
Theorem 1.
With the decomposition strategy, Eq.
(2) (i.e., the ELBO) naturally has the following factorized form:
[Equation]
The detailed inference process of the above formula is provided in Appendix A.1.
The ELBO is split into three main units, i.e., Eqs.
(3), (4), and (5).
The predictor makes forecasting and measures the accuracy (e.g,.
L1 or L2 losses), reconstruction and KL divergence are served as regularization terms aiming to improve the learned representations.
The three units are described in the following.
Predictor: The predictor (cf.
Eq.
(3) and Figure 1(b)) can be regarded as the sum of two independent parts: log (cid:82)
Z s P ψ s ( Y s | Z s ) Q ϕ s ( Z s | X ) dZ s and log (cid:82) Z t P ψ t ( Y t | Z t ) Q ϕ t ( Z t | X ) dZ t .
Here we introduce two specialized approaches to harness the seasonal-trend representations combining their own characteristics.
Given the seasonal latent representation Z s ∈ R T × d , the seasonal predictor first employs the discrete Fourier transform (DFT) algorithm to detect the seasonal frequencies, i.e.,
Z s F = DFT ( Z s ) ∈ C F × d , where F =
⌊ T +12 ⌋ due to the Nyquist theorem [42].
Then, we inverse the frequencies back to the temporal domain to extend the representation to the future part, i.e.,
˜ Z s = iDFT ( Z s F ) ∈ R τ × d .
More details of the DFT and iDFT functions can be found in Appendix
B.1.
Given Z t , trend predictor provides a feed forward network (FFN)
f : T → τ to produce a predictable representation
˜ Z t ∈ R τ × d .
We end the predictor with two FFNs to map
˜ Z s and ˜ Z t into
Y s and Y t , respectively, and obtain the forecasting result Y by their sum.
Reconstruction and KL divergence:
Among these two terms, the KL divergence can be easily estimated by Monte Carlo sampling with prior assumptions.
Here we take a widely used setting that priors both follow N (0 , I ) for efficiency, more discussions of our priors can be found in Appendix
C. As for reconstruction term, it cannot be directly measured owing to the unknown X s and X t .
Besides, merging these two terms into E Q ϕs,ϕt ( Z s ,Z t | X ) [log P θ s ,θ t
X | Z s , Z t )] will result in chaos since the decoder is prone to reconstruct the intricate time series from every representation.
Theorem 2.
With the Gaussian distribution assumption, the reconstruction loss L rec can be estimated without leveraging X s and X t , and Eq.
(4) can be replaced with the following formula:
[Equation]
where A XX ( κ ) = (cid:80) T − κ i [-=
-]=1 ( X t − ¯ X )( X t + κ − ¯ X ) is the autocorrelation coefficient with lagged value κ (we employ an efficient implementation in frequency domain [43], details are in Appendix B.2),
CORT ( X, ˆ X t ) is the temporal correlation coefficient, and ∆ X i =
X i − X
i − 1 is the first difference.
The proof is provided in Appendix A.2.
According to Eq.
(6), the reconstruction loss now can be estimated and, conversely, used to supervise disentangled representation learning.
However, we find that this framework still holds certain drawbacks: (1) The KL divergence tends to narrow down the distance between posterior and prior.
The modeling choice tends to sacrifice the variational inference vs. data fit when modeling capacity is not sufficient to achieve both [44].
The posterior may become almost non-informative for the inputs, which causes the forecastings irrelevant to the observations.
The disentanglement of the seasonal-trend representations is boosted indirectly by the separate reconstruction, where we need to impose a direct constraint on the representations themselves.
We alleviate these limitations by introducing additional mutual information regularization terms.
Specifically, we increase the mutual information between Z s , Z t and X to alleviate the divergence narrowing problem [44, 45], while decreasing mutual information between Z s and Z t to further dissociate their representations.
The maximizing objective of LaST becomes
[Equation]
where I ( · , · ) denotes the mutual information between two representations.
However, the two mutual information terms are untraceable [46, 47, 48].
We address this problem in the next section.
Mutual information bounds for optimization
We now address the traceable mutual information bounds, maximizing I ( X, Z s ) and I ( X, Z t ) , and minimizing I ( Z s , Z t ) in Eq.
(8), and provide lower and upper bounds for the model optimization.
Lower bound for I ( X, Z s ) or I ( X, Z t ) .
We omit the superscript s or t when analyzing lower bound.
Among the prior approaches exploring the lower bounds for MI [49, 50, 51], MINE [51], for example, employs KL divergence between the joint distribution and marginals and defines an energybased variational family to achieve a flexible and scalable lower bound.
This can be formulated as I ( X, Z ) ≥ E Q ϕ ( X,Z )
[ γ α ( X, Z )] − log E Q ( x ) Q [Equation] ( z ) [
[Equation]
α ( X,Z ) ]
= I MINE , where γ α is a learned normalized critic with parameters α .
However, this bound suffers from the biased gradient owing to the parametric logarithmic term (see Appendix A.3 for a proof).
Inspired by [47], we substitute the logarithmic function by its tangent family to ameliorate the above biased bound:
[Equation]
(9) where η denotes the different tangent points.
The first inequality relies on the concave negative logarithmic function – the values on the curve are upper bounds for that on the tangent line, and is tight when the tangent point overlaps the independent variable, i.e., the true value of E Q ( x ) Q ( z )
[ e γ ( X,Z ) ] .
The closer the distance between tangent point and independent variable, the greater the lower bound.
Therefore, we set η as the variational term E Q ( x ) Q ϕ ( z ) [ e γ α ( X,Z ) ] that estimates the independent variable to obtain as great lower bound as possible.
In the second inequality, γ α ( x, z ) – a critic function activated by Sigmoid – is limited within [0 , 1] and thus − (log η − 1) ≥ 0 .
This inequality is tight only if E Q ( x ) Q ϕ ( z )
[ γ α ( X, Z )]
= 1 , which means γ α can discriminate whether a pair of variables ( X, Z ) is sampled from the joint distribution or marginals.
Similarly to MINE, this consistency problem can be addressed by the universal approximation theorem for neural networks [52].
Thus,
Eq.
(9) provides a flexible and scalable lower bound for I ( X, Z ) with unbiased gradient.
For the evaluation, we exploit a traceable manner [53, 51] that draws joint samples ( X ( i ) , Z ( i ) ) by
Q ( Z ( i ) |
X ( i ) )
P D ( X ( i ) ) .
As for the marginal Q ϕ ( Z ) , we randomly select a datapoint j
and then sample it from Q ϕ
( Z | X ( j ) )
P D ( X ( j ) ) .
Details of the optimization are shown in Algorithm 1.
Upper bound for I ( Z s , Z t ) .
Few efforts have been made that explore the traceable upper bound for mutual information [54, 47, 55].
Existing upper bounds (listed in Appendix D.1) are traceable with known probabilistic density of joint or conditional distributions here being Q
( Z s | Z t ) , Q ( Z t | Z s ) or
Q ( Z s , Z t ) .
However, these distributions lack the interpretability and can hardly be directly modeled, which leads to untraceable estimations of the above upper bounds.
To avoid the direct estimation of unknown probabilistic densities, we introduce an energy-based variational family for Q ( Z s , Z t ) that uses a normalized critic γ β ( Z s , Z t ) like Eq.
(9) to establish a traceable upper bound.
Specifically, we incorporate the critic γ β into the upper bound I CLUB [55] to obtain a traceable Seasonal-Trend Upper Bound (STUB) for I ( Z s , Z t ) , which is defined as:
[Equation]
The derivation details of this formula are provided in Appendix D.2.
The inequality in Eq.
(10) is tight only if Z s and Z t are a pair of independent variables [55].
This is exactly a sufficient condition for I STUB , since MI and Eq.
(11) are both zeros on the independent situation, which is our seasonaltrend disentanglement optimal objective.
The critic γ β , similar to γ α , takes on the discriminating responsibility but provides converse scores, constraining the MI to a minimum.
However, Eq.
(11) may get negative values during the learning of parameter β , resulting an invalid upper bound for MI.
To alleviate this problem, we additionally introduce a penalty term ∥
I neg STUB ∥ 2 to assist the model optimization, which is a L2 loss of the negative parts in I STUB .
For the evaluation, we take the same sampling manner as the one in the lower bound and optimization details are also shown in Algorithm 1. [Table] [Table] [Table]
Update parameters Γ
[Table] [Table] i = j =
[Table]
[Table]
α
[Table]
( X ( i ) , Z s ( i ) )
[Table]
η e γ αs ( X ( i ) ,Z s ( j ) )
+
[Table]
i )
[Table]
i )
[Table] e [Table] αt
[Table]
i ) ,Z t [Table] j ) ) ) [Table]
B
[Table]
[Table] i ) , Z t [Table] i ) )
[Table]
Z s
[Table]
i ) , Z t
[Table]
j ) ) + average ( (cid:13)(cid:13) [Table] i ) ,
[Table]
i ) ) − [Table] i ) ,
[Table]
j ) )
[Table]
neg
[Table]
2 )] ; 8: end for
Experiments
We now present the results of our extensive experimental evaluations comparing LaST with state-ofthe-art baselines and report a series of empirical results, along with ablation study and visualizations of seasonal-trend representations.
Further details and results are provided in Appendix F.
5.1 Settings
Datasets and Baselines.
We conducted our experiments on seven real-world benchmark datasets from four categories of mainstream time series forecasting applications: (1) ETT 2 [25]: Electricity
Transformer Temperature consists of the target value “oil temperature” and six “power load” features, recorded hourly (i.e., ETTh1 and ETTh2) and every 15 minutes (i.e., ETTm1 and ETTm2) over two years.
(2) Electricity , from the UCI Machine Learning Repository 3 and preprocessed by [56], is composed of the hourly electricity consumption of 321 clients in kWh from 2012 to 2014.
Exchange [56] with daily exchange rates of eight countries from 1990 to 2016.
(4) Weather 4 contains 21 meteorological indicators (e.g., temperature and humidity) and is recorded every 10 minutes in 2020.
We compare our LaST with latest state-of-the-art methods on time series modeling and forecasting tasks from two categories: (1) representation learning techniques, including COST [38],
TS2Vec [37], and TNC [36]; (2) end-to-end forecasting models, including VAE-GRU [10], Autoformer [26], Informer [25], and TCN [22].
Further descriptions and settings of these baselines are provided in appendix F.1.
Evaluation setup.
Following the prior work, we run our model on both univariate and multivariate forecasting settings.
In multivariate forecasting, LaST accepts and forecasts all variables in datasets.
In univariate forecasting, LaST only consider a specific feature in each dataset.
We employ the standard normalization and set input length T = 201 for all datasets.
For the dataset split, we follow standard protocol that categorizes all datasets into training, validation and test set in chronological order by the ratio of 6:2:2 for all datasets.
We report the evaluation results on the test set while the model achieves the best performance on the validation set.
Implementation details.
As for the network structure of LaST, we use a single-layer fully connected network as the feed forward network (FFN), which is applied in the modeling of posterior, reconstruction, and predictor.
Besides, we employ the 2-layer MLP for the critic γ in MI bound estimations.
Dimensions of seasonal and trend representations are consistent.
We set them as 32 in univariate forecasting and as 128 in multivariate forecasting on other datasets.
MAE loss is used to measure
[Table]
the forecasting derived from the predictor.
For the training strategy, we use the Adam [57] optimizer and training process is early stopped within 10 epoches.
We initialize the learning rate with 10 - 3 and decay it with 0 .
95 weight every epoch.
5.2 Performance comparisons and model analysis
Effectiveness.
Tables 1 and 2 summarize the results of univariate and multivariate forecastings respectively.
LaST achieves state-of-the-art performance against the advanced representation baselines on five real-world datasets.
The relative improvements on MSE and MAE are 25 .
6% and 22 .
1% against the best representation learning method CoST
and are 22 . 0% and 18 .
9% against the best end-to-end models Autoformer.
We note that Autoformer achieves better performance on long horizons forecasting on hourly ETT datasets and think there are two reasons: (1) Transformer-based models intrinsically establish long-range dependencies, which plays a crucial role in long sequence forecasting; (2) it employs a simple decomposition by average pooling with a fixed kernel size, which is more suitable for strongly periodic dataset like hourly ETT.
This phenomenon is beneficial to long-term forecasting but limits the sensitivity to local context, and the bonus does not have significant impact on other datasets.
Compared with baselines, LaST extracts the seasonal and trend patterns with disentangled representations adaptively and thus can be applied to intricate time series.
Ablation study.
We investigated the performance benefits brought by each mechanism of LaST on a synthetic dataset (generation details are provided in appendix F.3) and
ETTh1.
The results are shown in Table 3, consisting of two groups: M1 validates the mechanisms of seasonal-trend representations learning framework.
In it, “w/o seasonal” and “w/o trend” denote LaST without the seasonal and trend components respectively; “w/o coe” denotes LaST without autocorrelation and CORT coefficients while estimating the reconstruction loss.
M2 judges the introduction and estimations of MI, where“w/o lower” and “w/o upper” indicate the removal of the lower and upper bounds for
MI in regularization terms respectively; “with MINE” denotes that we replace our lower bound with
MINE.
The results show that all mechanisms improve the performance on the forecasting task.
We notice that the quality drops a lot when removing the trend component.
The reason is the seasonal forecasting derives from the iDFT algorithm, which is essentially a periodical repetition of the history observations.
However, it captures the seasonal patterns and assists the trend component in complete
LaST to bring the superiority, especially in the long-term settings and strongly periodical synthetic dataset.
Besides, we observe that with biased regularization term MINE, the performance becomes unstable and sometimes even worse than LaST without MI lower bound, while our unbiased bound (cf. Eq.(9)) continuously outperforms it.
Representation disentanglement.
We visualize the seasonal-trend representations with the tSNE [58] technique in Figure 2.
We also visualize the embeddings in last layer of Autoformer decoder as a comparison.
The points with same color have a clearer and closer clustering in LaST, while they mix together without decomposition mechanisms (“w/o dec” indicates removal of the two decomposition mechanisms (autocorrelation and CORT coefficients, and the upper bound to MI).
[Table] [Table]
Notably, though Autofomer with the simple moving average block achieves satisfying decomposition from the time series perspective, their representations are still prone to entanglement.
These results suggest that (1) learning disentangled seasonal-trend representations is not trivial
[Table]
and (2) the proposed decomposition mechanisms successfully disentangle the seasonal-trend representations in latent space, each paying attention to a specific temporal pattern.
Input settings.
We further investigate the influence of hyperparameter input length to validate the sensitivity and Table 4 shows the results.
Long look-back window improves the performance especially in long-term forecasting
[Table]
while others even have performance degradation.
This verifies that LaST can effectively utilize the past information to understand patterns and make predictions
[Table]
[Table]
L a S
T w / o d ec
A u t o f o r m e r
L a S
T
A u t o f o r m e r
L a S
T
A u t o f o r m e r
L a S
T
A u t o f o r m e r
L a S
T
A u t o f o r m e r
L a S
T
A u t o f o r m e r
[Figure]
Observations from a case-study.
We further validate LaST by by visualizing the extracted seasonality and trend in specifica cases.
As shown in Figure 3, LaST can capture the seasonal patterns on realworld datasets.
For example, a strong daily period is indicated on hourly and 15-minutes ETT datasets.
Even the period on Exchange dataset is not obvious, LaST still provides some long-term periods on the daily data.
Besides, trend and seasonal components jointly accurately restore the original sequence with their own perspective, which supports that LaST can produce workable disentangled representations for intricate time series.
Conclusion
We presented LaST, a disentangled variational inference framework with mutual information constraints to disassociate a couple of seasonal-trend representations in latent space, for effective forecasting of time series.
Our extensive experiments demonstrated that LaST successfully disentangles the seasonal-trend representations and achieves state-of-the-art performance.
Our future work will focus on tackling other challenging downstream tasks in the time series domain, e.g., generation and imputation.
In addition, we plan to model stochastic factors explicitly in decomposition strategies, which will better understand the real-world time series.
Acknowledgments and Disclosure of Funding
This work was supported in part by National Natural Science Foundation of China (Grant
No
[Table]
For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
[Yes] (b) Did you describe the limitations of your work?
[No] (c) Did you discuss any potential negative societal impacts of your work?
[No] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes] 2.
If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
[Yes] (b) Did you include complete proofs of all theoretical results?
[Yes] See Appendix A. 3.
If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Our source code for reproducibility is publicly available at GitHub.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
[Yes] See “Datasets” and “Evaluation setup” in Sec. 5.1.
(c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
[Yes] See Appendix F.5.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix F.2.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
[Yes] See URLs in Sec.
(b) Did you mention the license of the assets?
[Yes] See folder “license” in supplementary materials.
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes] Our source code for reproducibility is included in the supplemental material.
(d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
[Yes] See folder “license” in supplementary materials.
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
[Yes] They do not contain personally identifiable information or offensive content.
If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
[N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
[N/A]