Deep Explicit Duration Switching Models for Time Series

Abstract:
Many complex time series can be effectively subdivided into distinct regimes that exhibit persistent dynamics. Discovering the switching behavior and the statistical patterns in these regimes is important for understanding the underlying dynamical system. We propose the Recurrent Explicit Duration Switching Dynamical System (RED-SDS), a flexible model that is capable of identifying both stateand time-dependent switching dynamics. State-dependent switching is enabled by a recurrent state-to-switch connection and an explicit duration count variable is used to improve the time-dependent switching behavior. We demonstrate how to perform efficient inference using a hybrid algorithm that approximates the posterior of the continuous states via an inference network and performs exact inference for the discrete switches and counts. The model is trained by maximizing a Monte Carlo lower bound of the marginal log-likelihood that can be computed efficiently as a byproduct of the inference routine. Empirical results on multiple datasets demonstrate that RED-SDS achieves considerable improvement in time series segmentation and competitive forecasting performance against the state of the art.

heading: 1 Introduction
text: Time series forecasting plays a key role in informing industrial and business decisions [17, 24, 8], while segmentation is useful for understanding biological and physical systems [40, 45, 34]. State Space Models (SSMs) [16] are a powerful tool for such tasks—especially when combined with neural networks [42, 12, 13]—since they provide a principled framework for time series modeling. One of the most popular SSMs is the Linear Dynamical System (LDS) [5, 43], which models the dynamics of the data using a continuous latent variable, called state, that evolves with Markovian linear transitions. The assumptions of LDS allow for exact inference of the states [27]; however, they are too restrictive for real-world systems that often exhibit piecewise linear or non-linear hidden dynamics with a finite number of operating modes or regimes. For example, the power consumption of a city may follow different hidden dynamics during weekdays and weekends. Such data are better explained by a Switching Dynamical System (SDS) [1, 21], an SSM with an additional set of latent variables called switches that define the operating mode active at the current timestep.
Switching events can be classified into time-dependent or state-dependent [33]. Historically, emphasis was placed on the former, which occurs after a certain amount of time has elapsed in a given regime. While in a vanilla SDS switch durations follow a geometric distribution, more complex long-term
∗Equal contribution. †Work done during an internship at Amazon Research.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).
temporal patterns can be captured using explicit duration models [40, 9]. As a recent alternative to time-dependency, recurrent state-to-switch connections [35] have been proposed that capture state-dependent switching, i.e., a change that occurs when the state variable enters a region that is governed by a different regime. For added flexibility, these models can be used in conjunction with transition/emission distributions parameterized by neural networks [25, 19, 13, 30]. Recent works, e.g., [13, 30], proposed hybrid inference algorithms that exploit the graphical model structure to perform approximate inference for some latent variables and conditionally exact inference for others.
Despite these advances in representation and inference, modeling complex real-world temporal phenomena remains challenging. For example, state-of-the-art statedependent models (e.g., [13]) lack the capacity to adequately capture time-dependent switching. Empirically, we find this hampers their ability to learn parsimonious segmentations when faced with complex patterns and long-term dependencies (see Fig. 1 for an example).
Conversely, time-dependent switching models are “open-loop” and unable to model state-conditional behavioral transitions that are common in many systems, e.g., in autonomous or multi-agent systems [35]. Intuitively, the suitability of the switching model largely depends on the underlying data-generating process; city power consumption may be better modeled via time-dependent switching, whilst the motion of a ball bouncing between two walls is driven by its state. Indeed, complex real-world processes likely involve both types of switching behavior.
Motivated by this gap, we propose the Recurrent Explicit Duration Switching Dynamical System (RED-SDS) that captures both state-dependent and time-dependent switching. RED-SDS combines the recurrent state-to-switch connection with explicit duration models for switches. Notably, REDSDS allows the incorporation of inductive biases via the hyperparameters of the duration models to better identify long-term temporal patterns. However, this combination also complicates inference, especially when using neural networks to model the underlying probability distributions. To address this technical challenge, we propose a hybrid algorithm that (i) uses an inference network for the continuous latent variables (states) and (ii) performs efficient exact inference for the discrete latent variables (switches and counts) using a forward-backward routine similar to Hidden Semi-Markov Models [48, 9]. The model is trained by maximizing a Monte Carlo lower bound of the marginal log-likelihood that can be efficiently computed by the inference routine.
We evaluated RED-SDS on two important tasks: segmentation and forecasting. Empirical results on segmentation show that RED-SDS is able to identify both state- and time-dependent switching patterns, considerably outperforming benchmark models. For example, Fig. 1 shows that RED-SDS addresses the oversegmentation that occurs with an existing strong baseline [13]. For forecasting, we illustrate the competitive performance of RED-SDS with an extensive evaluation against state-ofthe-art models on multiple benchmark datasets. Further, we show how our model is able to simplify the forecasting problem by breaking the time series into different meaningful regimes without any imposed structure. As such, we manage to learn appropriate duration models for each regime and extrapolate the learned patterns into the forecast horizon consistently.
In summary, the key contributions of this paper are:
• RED-SDS, a novel non-linear state space model which combines the recurrent state-toswitch connection with explicit duration models to flexibly model switch durations;
• an efficient hybrid inference and learning algorithm that combines approximate inference for states with conditionally exact inference for switches and counts;
• a thorough evaluation on a number of benchmark datasets for time series segmentation and forecasting, demonstrating that RED-SDS can learn meaningful duration models, identify both state- and time-dependent switching patterns and extrapolate the learned patterns consistently into the future.

heading: 2 Background: switching dynamical systems
text: Notation. Matrices, vectors and scalars are denoted by uppercase bold, lowercase bold and lowercase normal letters, respectively. We denote the sequence {y1, . . . ,yT } by y1:T , where yt is the value of y at time t. In our notation, we do not further differentiate between random variables and their realizations.
Switching Dynamical Systems (SDS) are hybrid SSMs that use discrete “switching” states zt to index one of K base dynamical systems with continuous states xt. The joint distribution factorizes as
p(y1:T ,x1:T , z1:T ) = T∏ t=1 p(yt|xt)p(xt|xt−1, zt)p(zt|zt−1), (1)
where p(x1|x0, z1)p(z1|z0) = p(x1|z1)p(z1) is the initial (continuous and discrete) state prior. The base dynamical systems have continuous state transition p(xt|xt−1, zt) and continuous or discrete emission p(yt|xt) that can both be linear or non-linear. The discrete transition p(zt|zt−1) of vanilla SDS is parametrized by a stochastic transition matrix A ∈ RK×K , where the entry aij = A(i, j) represents the probability of switching from state i to state j. This results in an “open loop” as the transition only depends on the previous switch which inhibits the model from learning state-dependent switching patterns [35]. Further, the state duration (also known as the sojourn time) follows a geometric distribution [9], where the probability of staying in state i for d steps is ρi(d) = (1− aii)ad−1ii . This memoryless switching process results in frequent regime switching, limiting the ability to capture consistent long-term time-dependent switching patterns. In the following, we briefly discuss two approaches that have been proposed to improve the state-dependent and time-dependent switching capabilities in SDSs.
Recurrent SDS. Recurrent SDSs (e.g., [6, 35, 7, 30]) address state-dependent switching by changing the switch transition distribution to p(zt|xt−1, zt−1)—called the state-to-switch recurrence— implying that the switch transition distribution changes at every step and the sojourn time no longer follows a geometric distribution. This extension complicates inference. Furthermore, the first-order Markovian recurrence does not adequately address long-term time-dependent switching.
Explicit duration SDS. Explicit duration SDSs are a family of models that introduce additional random variables to explicitly model the switch duration distribution. Explicit duration variables have been applied to both HMMs and SDSs with Gaussian linear continuous states; the resulting models are referred to as Hidden Semi-Markov Models (HSMMs) [38, 48], and Explicit Duration Switching Linear Gaussian SSMs (ED-SLGSSMs) [9, 40, 10], respectively. Several methods have been proposed in the literature for modeling the switch duration, e.g., using decreasing or increasing count, and duration-indicator variables. In the following, we briefly describe modeling switch duration using increasing count variables and refer the reader to Chiappa [9] for details.
Increasing count random variables ct represent the run-length of the currently active regime and can either increment by 1 or reset to 1. An increment indicates that the switch variable zt is copied over to the next timestep whereas a reset indicates a regular Markov transition using the transition matrix A. Each of the K switches has a distinct duration distribution ρk, a categorical distribution over {dmin, . . . , dmax}, where dmin and dmax delimit the number of steps before making a Markov transition. Following [40, 9], the probability of a count increment is given by
vk(c) = 1− ρk(c)∑dmax d=c ρk(d) . (2)
The transition of count ct and switch zt variables is defined as p(ct|zt−1 = k, ct−1) = { vk(ct−1) if ct = ct−1 + 1 1− vk(ct−1) if ct = 1 , (3)
p(zt = j|zt−1 = i, ct) = { δzt=i if ct > 1 A(i, j) if ct = 1 , (4)
where δcond denotes the delta function which takes the value 1 only when cond is true.
Although SDSs with explicit switch duration distributions can identify long-term time-dependent switching patterns, the switch transitions are not informed by the state—inhibiting their ability to
model state-dependent switching events. Furthermore, to the best of our knowledge, SDSs with explicit duration models have only been studied for Gaussian linear states [10, 9, 40].

heading: 3 Recurrent explicit duration switching dynamical systems
text: In this section we describe the Recurrent Explicit Duration Switching Dynamical System (RED-SDS) that combines both state-to-switch recurrence and explicit duration modeling for switches in a single non-linear model. We begin by formulating the generative model as a recurrent switching dynamical system that explicitly models the switch durations using increasing count variables. We then discuss how to perform efficient inference for different sets of latent variables. Finally, we discuss how to estimate the parameters of RED-SDS using maximum likelihood.

heading: 3.1 Model formulation
text: Consider the graphical model in Fig. 2 (a); the joint distribution of the counts ct ∈ {1, . . . , dmax}, the switches zt ∈ {1, . . . ,K}, the states xt ∈ Rm, and the observations yt ∈ Rd, conditioned on the control inputs ut ∈ Rc, factorizes as
pθ(y1:T ,x1:T , z1:T , c1:T |u1:T ) = p(y1|x1)p(x1|z1,u1)p(z1|u1)
· [ T∏ t=2 p(yt|xt)p(xt|xt−1, zt,ut)p(zt|xt−1, zt−1, ct,ut)p(ct|zt−1, ct−1,ut) ] .
(5)
Similar to [40, 9], we consider increasing count variables ct to incorporate explicit switch durations into the model, i.e., ct can either increment by 1 or reset to 1 at every timestep and represent the run-length of the current regime. A self-transition is allowed after the exhaustion of dmax steps for flexibility. In the subsequent discussion we omit the control inputs ut for clarity of exposition.
We model the initial prior distributions in Eq. (5) for the respective discrete and continuous case as
p(z1) = Cat(z1;π), (6) p(x1|z1) = N (x1;µz1 ,Σz1), (7)
where Cat denotes a categorical and N a multivariate Gaussian distribution. The transition distributions for the discrete variables (count and switch) are given by
p(ct|zt−1, ct−1) = { vzt−1(ct−1) if ct = ct−1 + 1 1− vzt−1(ct−1) if ct = 1 , (8)
p(zt|xt−1, zt−1, ct) = { δzt=zt−1 if ct > 1 Cat(zt;Sτ (fz(xt−1, zt−1))) if ct = 1 , (9)
where Sτ is the tempered softmax function (cf. Section 3.3) with temperature τ , and fz can be a linear function or a neural network. The probability of a count increment vk for a switch k is defined via the duration model ρk as in Eq. (2). The continuous state transition and the emission are given by
p(xt|xt−1, zt) = N (xt; fµx (xt−1, zt), fΣx (xt−1, zt)), (10) p(yt|xt) = N (yt; fµy (xt), fΣy (xt)), (11)
where fµx , f Σ x , f µ y , f Σ y are again linear functions or neural networks.
The model is general and flexible enough to handle both state- and time-dependent switching. The switch transitions zt−1 → zt are conditioned on the previous state xt−1 which ensures that the switching events occur in a “closed loop”. The switch duration models ρk provide flexibility to stay long term in the same regime, allowing to better capture time-dependent switching. We use increasing count variables to incorporate switch durations into our model as they are more amenable to the case when the count transitions depend on the control ut. For instance, decreasing count variables, another popular option [11, 36, 9], deterministically count down from the sampled segment duration length to 1. This makes it difficult to condition the switch duration model on the control inputs. In contrast, increasing count variables increment or reset probabilistically at every timestep.

heading: 3.2 Inference
text: Exact inference is intractable in SDSs and scales exponentially with time [32]. Various approximate inference procedures have been developed for traditional SDSs [14, 21, 6], while more recently inference networks have been used for amortized inference for all or a subset of latent variables [25, 28, 13, 30]. Particularly, Dong et al. [13] used an inference network for the states and performed exact HMM-like inference for the switches, conditioned on the states. We take a similar approach and use an inference network for the continuous latent variables (states) and perform conditionally exact inference for the discrete latent variables (switches and counts) similar to the forward-backward procedure for HSMM [48, 9]. We define the variational approximation to the true posterior p(x1:T , z1:T , c1:T |y1:T ) as q(x1:T , z1:T , c1:T |y1:T ) = qφ(x1:T |y1:T )pθ(z1:T , c1:T |y1:T ,x1:T ) where φ and θ denote the parameters of the inference network and the generative model respectively.
Approximate inference for states. The posterior distribution of the states, qφ(x1:T |y1:T ), is approximated using an inference network. We first process the observation sequence y1:T using a non-causal network such as a bi-RNN or a Transformer [46] to simulate smoothing by incorporating both past and future information. The non-causal network returns an embedding of the data h11:T which is then fed to a causal RNN that outputs the posterior distribution qφ(x1:T |y1:T ) = ∏ t q(xt|x1:t−1,h11:T ). See Fig. 2 (b) for an illustration of the inference procedure.
Exact inference for counts and switches. Inference for the switches z1:T and the counts c1:T can be performed exactly conditioned on states x1:T and observations y1:T . Samples from the approximate posterior x̃1:T ∼ q(x1:T |y1:T ) are used as pseudo-observations of x1:T to infer the posterior distribution pθ(z1:T , c1:T |y1:T , x̃1:T ). A naive approach to infer this distribution is by treating the pair (ct, zt) as a “meta switch” that takesKdmax possibles values and perform HMM-like forward-backward inference. However, this results in a computationally expensive O(TK2d2max) procedure that scales poorly with dmax. Fortunately, we can pre-compute some terms in the forwardbackward equations by exploiting the fact that the count variable can only increment by 1 or reset to 1 at every timestep. This results in an O(TK(K + dmax)) algorithm that scales gracefully with dmax [9]. The forward αt and backward βt variables, defined as
αt(zt, ct) = p(y1:t,x1:t, zt, ct), (12) βt(zt, ct) = p(yt+1:T ,xt+1:T |xt, zt, ct), (13)
can be computed by modifying the forward-backward recursions used for the HSMM [9] to handle the additional observed variables x1:t. We refer the reader to Appendix A.1 for the exact derivation.

heading: 3.3 Learning
text: The parameters {φ, θ} can be learned by maximizing the evidence lower bound (ELBO): LELBO = Eq(x1:T |y1:T )p(z1:T ,c1:T |y1:T ,x1:T ) [ log p(y1:T ,x1:T , z1:T , c1:T , )
q(x1:T |y1:T )p(z1:T , c1:T |y1:T ,x1:T ) ] = Eq(x1:T |y1:T ) [ log p(y1:T ,x1:T )
q(x1:T |y1:T )
] .
(14)
The likelihood term p(y1:T ,x1:T ) can be computed using the forward variable αT (zT , cT ) by marginalizing out the switches and the counts,
p(y1:T ,x1:T ) = ∑ zT ,cT αT (zT , cT ), (15)
and the entropy term −Eq(x1:T |y1:T ) [log q(x1:T |y1:T )] can be computed using the approximate posterior q(x1:T |y1:T ) output by the inference network. The ELBO can be maximized via stochastic gradient ascent given that the posterior q(x1:T |y1:T ) is reparameterizable. We note that Dong et al. [13] used a lower bound for the likelihood term in Switching Non-Linear Dynamical Systems (SNLDS); however, it can be computed succinctly by marginalizing out the discrete random variable (i.e., the switch in SNLDS) from the forward variable αT , similar to Eq. (15). Using our objective function, we observed that the model was less prone to posterior collapse (where the model ends up using only one switch) and we did not require the additional ad-hoc KL regularizer used in Dong et al. [13]. Please refer to Appendix B.4 for a brief discussion on the likelihood term in SNLDS.
Temperature annealing. We use the tempered softmax function Sτ to map the logits to probabilities for the switch transition p(zt|xt−1, zt−1, ct = 1) and the duration models ρk(d) which is defined as
Sτ (o)i = exp
( oi τ )∑ j exp ( oj τ
) , (16) where o is a vector of logits. The temperature τ is deterministically annealed from a high value during training. The initial high temperature values soften the categorical distribution and encourage the model to explore all switches and durations. This prevents the model from getting stuck in poor local minima that ignore certain switches or longer durations which might explain the data better.

heading: 4 Related work
text: The most relevant components of RED-SDS are recurrent state-to-switch connections and the explicit duration model, enabling both for state- and time-dependent switching. Additionally, RED-SDS allows for efficient approximate inference (analytic for switches and counts), despite parameterizing the various conditional distributions through neural networks. Existing methods address only a subset of these features as we discuss in the following.
The most prominent SDS is the Switching Linear Dynamical System (SLDS), where each regime is described by linear dynamics and additive Gaussian noise. A major focus of previous work has been on efficient approximate inference algorithms that exploit the Gaussian linear substructure (e.g., [21, 49, 14]). In contrast to RED-SDS, these models lack recurrent state-to-switch connections and duration variables and are limited to linear regimes.
Previous work has addressed the state-dependent switching by introducing a connection to the continuous state of the dynamical system [6, 35, 7, 30]. The additional recurrence complicates inference w.r.t. the continuous states; prior work uses expensive sampling methods in order to approximate the corresponding integrals [6] or as part of a message passing algorithm for joint inference of states and parameters [35]. On the other hand, ARSGLS [30] avoids sampling the continuous states by using conditionally linear state-to-switch connections and softmax-transformed Gaussian switch variables. However, both the ARSGLS and the related KVAE [19] can be interpreted as an SLDS with “soft” switches that interpolate linear regimes continuously rather than truly discrete states. This makes them less suited for time series segmentation compared to RED-SDS. Contrary to
the aforementioned models, RED-SDS allows non-linear regimes described by neural networks and incorporates a discrete explicit duration model without complicating inference w.r.t. the continuous states, since closed-form expressions are used for the discrete variables instead. Using amortized variational inference for continuous variables and analytic expressions for discrete variables has been proposed previously for segmentation in SNLDS [13]. RED-SDS extends this via an additional explicit duration variable that represents the run-length for the currently active regime.
Explicit duration variables have previously been proposed for changepoint detection [2, 3] and segmentation [10, 26]. For instance, BOCPD [2] is a Bayesian online changepoint detection model with explicit duration modeling. RED-SDS improves upon BOCPD by allowing for segment labeling rather than just detecting changepoints. The HDP-HSMM [26] is a Bayesian non-parametric extension to the traditional HSMM. Recent work [11, 36] has also combined HSMM with RNNs for amortized inference. These models—being variants of HSMM—do not model the latent dynamics of the data like RED-SDS. Chiappa and Peters [10] proposed approximate inference techniques for a variant of SLDS with explicit duration modeling. In contrast, RED-SDS is a more general non-linear model that allows for efficient amortized inference—closed-form w.r.t. the discrete latent variables.

heading: 5 Experiments
text: In this section, we present empirical results on two prominent time series tasks: segmentation and forecasting. Our primary goals were to determine if RED-SDS (a) can discover meaningful switching patterns in the data in an unsupervised manner, and (b) can probabilistically extrapolate a sequence of observations, serving as a viable generative model for forecasting. In the following, we discuss the main results and relegate details to the appendix.
5.1 Segmentation
We experimented with two instantiations of our model: RED-SDS (complete model) and ED-SDS, the ablated variant without state-to-switch recurrence. We compared against the closely related SNLDS [13] trained with a modified objective function. The original objective proposed in [13] suffered from training difficulties: it resulted in frequent posterior collapse and was sensitive to the crossentropy regularization term. Our version of SNLDS can be seen as a special case of RED-SDS with dmax = 1, i.e., without the explicit duration modeling (cf. Appendix B.4). We also conducted preliminary experiments on soft-switching models: KVAE [19] and ARSGLS [30]. However, these models use a continuous interpolation of the different operating modes which cannot always be correctly assigned to a single discrete mode, hence we do not report these unfavorable findings here (cf. Appendix B.4). For all models, we performed segmentation by taking the most likely value of the switch at
each timestep from the posterior distribution over the switches. As the segmentation labels are arbitrary and may not match the ground truth labels, we evaluated the models using multiple metrics: frame-wise segmentation accuracy (after matching the labelings using the Hungarian algorithm [29]), Normalized Mutual Information (NMI) [47], and Adjusted Rand Index (ARI) [23] (cf. Appendix B.2).
We conducted experiments on three benchmark datasets: bouncing ball, 3 mode system, and dancing bees to investigate different segmentation capabilities of the models. We refer the reader to Appendix B.1 for details on how these datasets were generated/preprocessed. For all the datasets, we set the number of switches equal to the number of ground truth operating modes.
Bouncing ball. We generated the bouncing ball dataset similar to [13], which comprises univariate time series that encode the location of a ball bouncing between two fixed walls with a constant velocity and elastic collisions. The underlying system switches between two operating modes (going up/down) and the switching events are completely governed by the state of the ball, i.e., a switch occurs only when the ball hits a wall. As such, the switching events are best explained by state-toswitch recurrence. All models are able to segment this simple dataset well as shown qualitatively in Fig 3 (a) and quantitatively in Table 1. We note that despite the seemingly qualitative equivalence, models with state-to-switch recurrence perform best quantitatively. RED-SDS learns to ignore the duration variable by assigning almost all probability mass to shorter durations (cf. Appendix B.5), which is intuitive since the recurrence best explains this dataset.
3 mode system. We generated this dataset from a switching linear dynamical system with 3 operating modes and an explicit duration model for each mode (shown in Fig. 4 (a)). We study this dataset in the context of time-dependent switching—the operating mode switches after a specific amount of time elapses based on its duration model. Both ED-SDS and RED-SDS learn to segment this dataset almost perfectly as shown in Fig. 3 (b) and Table 1 owing to their ability to explicitly model switch durations. In contrast, SNLDS fails to completely capture the long-term temporal patterns, resulting in spurious short-term segments as shown in Fig. 3 (b). Moreover, RED-SDS is able to recover the duration models associated with the different modes (Fig. 4). These results demonstrate that explicit duration models can better identify the time-dependent switching patterns in the data and can leverage prior knowledge about the switch durations imparted via the dmin and dmax hyperparameters.
Dancing bees. We used the publiclyavailable dancing bees dataset [40]—a challenging dataset that exhibits longterm temporal patterns and has been studied previously in the context of time series segmentation [41, 39, 18]. The dataset comprises trajectories of six dancer honey bees performing the waggle dance. Each trajectory consists of the 2D coordinates and the heading angle of a bee at every timestep with three possible types of motion: waggle, turn right, and turn left. Fig. 3 (c) shows that RED-SDS is able to segment the complex longterm motion patterns quite well. In contrast, ED-SDS identifies the long segment durations but often infers the mode inaccurately while SNLDS strug-
gles to learn the long-term motion patterns resulting in oversegmentation. This limitation of SNLDS is particularly apparent in the “waggle” phase of the dance which involves rapid, shaky motion. We also observed that sometimes ED-SDS and RED-SDS combined the turn right and turn left motions into a single switch, effectively segmenting the time series into regular (turn right and turn left) and waggle motion. This results in another reasonable segmentation, particularly in the absence of ground-truth supervision. We thus reevaluated the results after combining the turn right and turn left labels into a single label and present these results under dancing bees(K=2) in Table 1. Empirically, RED-SDS significantly outperforms ED-SDS and SNLDS on both labelings of the dataset. This suggests that real-world phenomena are better modeled by a combination of state- and time-dependent modeling capacities via state-to-switch recurrence and explicit durations, respectively.

heading: 5.2 Forecasting
text: We evaluated RED-SDS in the context of time series forecasting on 5 popular public datasets available in GluonTS [4], following the experimental set up of [30]. The datasets have either hourly or daily frequency with various seasonality patterns such as daily, weekly, or composite. In Appendix C.1 we provide a detailed description of the datasets. We compared RED-SDS to closely related forecasting models: ARSGLS and its variant RSGLS-ISSM [30]; KVAE-MC and KVAE-RB, which refer to the original KVAE [19] and its Rao-Blackwellized variant (as described in [30]) respectively; DeepState [42]; and DeepAR [44], a strong discriminative baseline that uses an autoregressive RNN (cf. Appendix C.4 for a discussion on these baselines).
We used data prior to a fixed forecast date for training and test the forecasts on the remaining unseen data; the probabilistic forecasts are conditioned on the training range and computed with 100 samples for each method. We used a forecast window of 150 days and 168 hours for datasets with daily and hourly frequency, respectively. We evaluated the forecasts using the continuous ranked probability score (CRPS) [37], a proper scoring rule [22] (cf. Appendix C.2). The results are reported in Table 2; RED-SDS compares favorably or competitively to the baselines on 4 out of 5 datasets.
Figure 5 illustrates how RED-SDS can infer meaningful switching patterns from the data and extrapolate the learned patterns into the future. It perfectly reconstructs the past of the time series and segments it in an interpretable manner without an imposed
seasonality structure, e.g., as used in DeepState and RSGLS-ISSM. The same switching pattern is consistently predicted into the future, simplifying the forecasting problem by breaking the time series into different regimes with corresponding properties such as trend or noise variance. Further, the duration models at several timesteps (the duration model is conditioned on the control ut) indicate that the model has learned how long each regime lasts and therefore avoids oversegmentation which would harm the efficient modeling of each segment. Notably, the model learns meaningful regime durations that sum up to the 24-hour day/night period for both K = 2 and K = 3 switches. Thus, RED-SDS brings the added benefit of interpretability—both in terms of the discrete operating mode and the segment durations—while obtaining competitive quantitative performance relative to the baselines.

heading: 6 Conclusion and future work
text: Many real-world time series exhibit prolonged regimes of consistent dynamics as well as persistent statistical properties for the durations of these regimes. By explicitly modeling both state- and time-dependent switching dynamics, our proposed RED-SDS can more accurately model such data. Experiments on a variety of datasets show that RED-SDS—when equipped with an efficient inference algorithm that combines amortized variational inference with exact inference for continuous and discrete latent variables—improves upon existing models on segmentation tasks, while performing similarly to strong baselines for forecasting.
One current challenge of the proposed model is that learning interpretable segmentation sometimes requires careful hyperparameter tuning (e.g., dmin and dmax). This is not surprising given the flexible nature of the neural networks used as components in the base dynamical system. A promising future research direction is to incorporate simpler models that have a predefined structure, thus exploiting domain knowledge. For instance, many forecasting models such as DeepState and RSGLS-ISSM parametrize classical level-trend and seasonality models in a non-linear fashion. Similarly, simple forecasting models with such structure could be used as base dynamical systems along with more flexible neural networks. Another interesting application is semi-supervised time series segmentation. For timesteps where the correct regime label is known, it is straightforward to condition on this additional information rather than performing inference; this may improve segmentation accuracy while providing an inductive bias that corresponds to an interpretable segmentation.
Funding disclosure
This work was funded by Amazon Research. This research is supported in part by the National Research Foundation Singapore under its AI Singapore Programme (Award Number: AISG-RP-2019011) to H. Soh.

Review summaries:
The authors propose Recurrent Explicit Duration Switching Dynamical System model, that is used to detect state and time switching dynamics. The method falls to the category of generalized switching state space model (Ghahramani & Hinton 2000). In particular, they use recurrent state-to-state switching with explicit duration models and provide approximate inference algorithm that is using evidence lower bound. Authors verify they model on 3 segmentation task from time series with ground truth and forecasting task on 5 datasets.