Original: B AYESIAN T IME S ERIES F ORECASTING WITH C HANGE P OINT AND A NOMALY D ETECTION
Revised: B AYESIAN T IME S ERIES F ORECASTING WITH C HANGE P OINT AND A NOMALY D ETECTION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Time series forecasting plays a crucial role in marketing, ﬁnance and many other quantitative ﬁelds.
Revised: Time series forecasting plays a crucial role in marketing, ﬁnance and many other quantitative ﬁelds.

Original: A large amount of methodologies has been developed on this topic, including ARIMA, Holt–Winters, etc.
Revised: A large amount of methodologies has been developed on this topic, including ARIMA, Holt–Winters, etc.

Original: However, their performance is easily undermined by the existence of change points and anomaly points, two structures commonly observed in real data, but rarely considered in the aforementioned methods.
Revised: However, their performance is easily undermined by the existence of change points and anomaly points, two structures commonly observed in real data, but rarely considered in the aforementioned methods.

Original: In this paper, we propose a novel state space time series model, with the capability to capture the structure of change points and anomaly points, as well as trend and seasonality.
Revised: In this paper, we propose a novel state space time series model, with the capability to capture the structure of change points and anomaly points, as well as trend and seasonality.

Original: To infer all the hidden variables, we develop a
Revised: To infer all the hidden variables, we develop a

Original: Bayesian framework, which is able to obtain distributions and forecasting intervals for time series forecasting, with provable theoretical properties.
Revised: Bayesian framework, which is able to obtain distributions and forecasting intervals for time series forecasting, with provable theoretical properties.

Original: For implementation, an iterative algorithm with Markov chain Monte Carlo (MCMC), Kalman ﬁlter and Kalman smoothing is proposed.
Revised: For implementation, an iterative algorithm with Markov chain Monte Carlo (MCMC), Kalman ﬁlter and Kalman smoothing is proposed.

Original: In both synthetic data and real data applications, our methodology yields a better performance in time series forecasting compared with existing methods, along with more accurate change point detection and anomaly detection.
Revised: In both synthetic data and real data applications, our methodology yields a better performance in time series forecasting compared with existing methods, along with more accurate change point detection and anomaly detection.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Time series forecasting has a rich and luminous history, and is essentially important in most of business operations nowadays.
Revised: Time series forecasting has a rich and luminous history, and is essentially important in most of business operations nowadays.

Original: The main aim of time series forecasting is to carefully collect and rigorously study the past observations of a time series to develop an appropriate model which could describe the inherent structure of the series, in order to generate future values.
Revised: The main aim of time series forecasting is to carefully collect and rigorously study the past observations of a time series to develop an appropriate model which could describe the inherent structure of the series, in order to generate future values.

Original: For instance, some internet companies may be interested in the number of daily active users (DAU), say, what is DAU after certain period of time, or when will reach their target DAU goal.
Revised: For instance, the internet companies are interested in the number of daily active users (DAU), say, what is DAU after certain period of time, or when will reach their target DAU goal.

Original: Time series forecasting is a fruitful research area with many existing methodologies.
Revised: Time series forecasting is a fruitful research area with many existing methodologies.

Original: The most popular and frequently used time series model might be the Autoregressive Integrated Moving Average (ARIMA)
Revised: The most popular and frequently used time series model might be the Autoregressive Integrated Moving Average (ARIMA)

Original: (Box et al., 2015; Zhang, 2003; Cochrane, 2005; Hipel & McLeod, 1994).
Revised: (Box et al., 2015; Zhang, 2003; Cochrane, 2005; Hipel & McLeod, 1994).

Original: Taking seasonality into consideration, Box et al.
Revised: Taking seasonality into consideration, Box et al.

Original: (2015) proposed the Seasonal ARIMA.
Revised: (2015) proposed the Seasonal ARIMA.

Original: The Holt–Winters method (Winters, 1960) is also very popular by using exponential smoothing.
Revised: The Holt–Winters method (Winters, 1960) is also very popular by using exponential smoothing.

Original: State space model (Durbin & Koopman, 2012; Scott & Varian, 2014; Brodersen et al., 2015) also attracts much attention, which is a linear function of an underlying Markov process plus additive noise.
Revised: State space model (Durbin & Koopman, 2012; Scott & Varian, 2014; Brodersen et al., 2015) also attracts much attention, which is a linear function of an underlying Markov process plus additive noise.

Original: ExponentialSmoothing State
Revised: Exponential

Original:  Space Model (ETS) (Hyndman et al., 2008) decomposes times series into error, trend, seasonal that change over time.
Revised: Smoothing State Space Model (ETS) (Hyndman et al., 2008) decomposes times series into error, trend, seasonal that change over time.

Revised: Recently, deep learning is applied for time-series trend learning using LSTM (Tao Lin, 2017), bidirectional dynamic Boltzmann machine (Osogami et al., 2017) is applied for time-series long-term dependency learning, and coherent probabilistic forecast (Taieb et al., 2017) is proposed for a hierarchy or an aggregation-level comprising a set of time series.

Revised: Orthogonal to these works, this paper focuses on robust ways of time series forecasting in presence of change points and anomalies.

Original: In Internet industry, Google develops the Bayesian structure time series (BSTS) model (Brodersen et al., 2015; Scott & Varian, 2014) to capture the trend, seasonality, and similar components of the target series.
Revised: In Internet time series forecasting, Google develops the Bayesian structure time series (BSTS) model (Brodersen et al., 2015; Scott & Varian, 2014) to capture the trend, seasonality, and similar components of the target series.

Original: Recently, Facebook proposes the Prophet approach (Taylor & Letham, 2017) based on a decomposable model with interpretable parameters that can be intuitively adjusted by analyst.
Revised: Recently, Facebook proposes the Prophet approach (Taylor & Letham, 2017) based on a decomposable model with interpretable parameters that can be intuitively adjusted by analyst.

Original: However, as in the DAU example, some special events like Christmas Holiday or President Election, newly launched apps or features, may cause short period or long-term change of DAU, leading to weird forecasting of those traditional models.
Revised: However, as in the DAU example, some special events like Christmas Holiday or President Election, newly launched apps or features, may cause short period or long-term change of DAU, leading to weird forecasting of those traditional models.

Original: The aforementioned special cases are well known as
Revised: The aforementioned special cases are well known as

Original: • Anomaly points.
Revised: • Anomaly points.

Original: The items, events or observations that don’t conform to an expected pattern or other items in the dataset, leading to a sudden spike or decrease in the series.
Revised: The items, events or observations that don’t conform to an expected pattern or other items in the dataset, leading to a sudden spike or decrease in the series.

Original: • Change points.
Revised: • Change points.

Original: A market intervention, such as a new product launch or the onset of an ads campaign, may lead to the level change of the original series.
Revised: A market intervention, such as a new product launch or the onset of an advertising (or ad) campaign, may lead to the level change of the original series.

Original: Time series forecasting without change/anomaly point detection and adjustment may also lead to bizarre forecasting since these models might learn the abrupt changes in the past.
Revised: Time series forecasting without change/anomaly point detection and adjustment may also lead to bizarre forecasting since these models might learn the abrupt changes in the past.

Original: There are literatures on detecting anomaly or change points individually, examples can be found in Twitter (2017);
Revised: There are literatures on detecting anomaly or change points individually, examples can be found in Twitter (2017);

Original: Netﬂix (2017); Barry & Hartigan (1993); Killick & Eckley (2014); twitter (2017).
Revised: Netﬂix (2017); Barry & Hartigan (1993); Killick & Eckley (2014); twitter (2017).

Original: However, the aforementioned change point detection models could not support detection in the presence of seasonality, while the presence of trend/change point is not handled by the anomaly detection models.
Revised: However, the aforementioned change point detection models could not support detection in the presence of seasonality, while the presence of trend/change point is not handled by the anomaly detection models.

Original: Most importantly, there is a discrepancy between anomaly/change points detection and adjustment, and commonly used manually adjustment might be a bit arbitrary.
Revised: Most importantly, there is a discrepancy between anomaly/change points detection and adjustment, and commonly used manually adjustment might be a bit arbitrary.

Original:  In this paper,  we develop a state space time series forecasting model in the Bayesian framework, jointly detect anomaly and change points.
Revised: Unfortunately, the forecasting gap caused by abnormal and change points, to the best of our knowledge, has not been given full attention and no good solution has been found so far. This paper is strongly motivated by bridging this gap. In this paper, to overcome the limitations of the most (if not all) current models that the anomaly points and change points are not properly considered, we develop a state space time series forecasting model in the Bayesian framework that can simultaneously detect anomaly and change points and perform forecasting. The learned structure information related to anomaly and change points is automatically incorporated into the forecasting process, which naturally enhances the model prediction based on the feedback of state-space model.

Original: For implementation, an iterative algorithm  with Markov chain Monte Carlo (MCMC), Kalman ﬁlter and Kalman smoothing is proposed.
Revised: To solve the resultant optimization problem, an iterative algorithm based on Bayesian approximate inference with Markov chain Monte Carlo (MCMC), Kalman ﬁlter and Kalman smoothing is proposed.

Original: The novel model could  capture the structure of change points, anomaly points, trend and seasonality, as also provide the distributions and forecasting intervals for time series forecasting.
Revised: The novel model could explicitly capture the structure of change points, anomaly points, trend and seasonality, as also provide the distributions and forecasting intervals due to Bayesian forecasting framework.

Original: Both synthetic and real data sets show the better performance of proposed model, in comparison with existing baseline.
Revised: Both synthetic and real data sets show the better performance of proposed model, in comparison with existing baseline.

Original: Moreover, our proposed model outperforms state-of-the-art models in identifying anomaly and change points.
Revised: Moreover, our proposed model outperforms state-of-the-art models in identifying anomaly and change points.

Original: To summarize, our work has the following contributions.
Revised: To summarize, our work has the following contributions

Original: • Our proposed method outperforms the state-of-the-art methods in time series forecasting, especially when there exist change points and anomalies.
Revised: • We proposed a robust 1 Bayesian state-space time series forecasting model that is able to explicitly capture the structures of change points and anomalies (which are generally ignored in most current models), and therefore automatically adapt for forecasting by incorporating the prior information of trend, seasonality, as well as change points and anomalies using state space modeling.

Original: By our method, we are able to obtain distributions and intervals for forecasting.
Revised: Due to the enhancement of model description capability, the results of model prediction and abnormal and change points detection are mutually improved. • To solve the resultant optimization problem, an effective algorithm based on approximate inference using Markov chain Monte Carlo (MCMC) is proposed with theoretical guaranteed forecasting paths.

Original: • Along with time series forecasting, our proposed method automatically detects change points and anomalies, and it achieves high accuracy and low false discovery rate on both tasks, outperformingsome popular change point and anomaly detection methods.
Revised: • Our proposed method outperforms the state-of-the-art methods in time series forecasting in presence of change points and anomalies, and detects change points and anomalies with high accuracy and low false discovery rate on both tasks, outperforming popular change point and anomaly detection methods.

Original: • Our method is ﬂexible to capture the structure of time series under various scenarios. By default, it takes the trend, seasonality, change points and anomalies into consideration, but it can be easily modiﬁed to study time series without some components, for example, time series without seasonality.
Revised: Our method is ﬂexible to capture the structure of time series under various scenarios with any component combinations of trend, seasonality, change points and anomalies.

Original: Thus, our method can be applied to many settings in practice
Revised: Therefore our method can be applied in many settings in practice.

Original:  M ODEL 
Revised:  M ODEL O VERVIEW

Original: State space time series model  has been one of the most popular models in time series analysis.
Revised: State space time series model (Hangos et al., 2014) has been one of the most popular models in time series analysis.

Original: It is capable of ﬁtting complicated time series structure including linear trend and seasonality.
Revised: It is capable of ﬁtting complicated time series structure including linear trend and seasonality.

Original: However, times series observed in real life are almost all prevailed with outliers.
Revised: However, times series observed in real life are almost all prevailed with outliers.

Original: Change points, less in frequency but are still widely observed in real time series analysis.
Revised: Change points, less in frequency but are still widely observed in real time series analysis.

Original: Unfortunately, both structures are ignored in the classic state space time series model.
Revised: Unfortunately, both structures are ignored in the classic state space time series model.

Original: In the section, we aim to address this issue by introducing a novel state space time series model.
Revised: In the section, we aim to address this issue by introducing a novel state space time series model.

Original: [Figure]
Revised: [Figure]

Original: Let y =
Revised: Let y =

Original: ( y 1 , y 2 , . . .
Revised: ( y 1 , y 2 , . . .

Original: , y n ) be a sequence of time series observations with length n .
Revised: , y n ) be a sequence of time series observations with length n .

Original: The ultimate goal is to forecast ( y n +1 , y n +2 , . . . ) .
Revised: The ultimate goal is to forecast ( y n +1 , y n +2 , . . . ) .

Original: The accuracy in forecasting lies in a successful decomposition of y into existing components.
Revised: The accuracy in forecasting lies in a successful decomposition of y into existing components.

Original: Apart from the residuals, we assume the time series is composed by trend, seasonality, change points and anomaly points.
Revised: Apart from the residuals, we assume the time series is composed by trend, seasonality, change points and anomaly points.

Original: In a nutshell, we have an additive model with time series =
Revised: In a nutshell, we have an additive model with time series =

Original: trend + seasonality + change point + anomaly point + residual.
Revised: trend + seasonality + change point + anomaly point + residual.

Original: Figure 1 provides a demonstration of desired decomposition of time series.
Revised: Figure 1 provides a demonstration of desired decomposition of time series.

Original: In Figure 1, thetop left panel shows the observed time series.
Revised: In Figure 1, the left panel shows the observed time series.

Original: And it can be decomposed into the remaining ﬁve panels.
Revised: And it can be decomposed into the remaining ﬁve panels.

Original: The shift in the change point panel shows where the change point lies.
Revised: The shift in the change point panel shows where the change point lies.

Original: And the spikes in the last panel reveals the anomaly points.
Revised: And the spikes in the last panel reveals the anomaly points.

Original: As the classical state space model, we have observation equation and transition equations to model y and hidden variables.
Revised: As the classical state space model, we have observation equation and transition equations to model y and hidden variables.

Original: We use µ = ( µ 1 , µ 2 , . . . , µ n ) to model trend, and use γ = ( γ 1 , γ 2 , . .
Revised: We use µ = ( µ 1 , µ 2 , . . . , µ n ) to model trend, and use γ = ( γ 1 , γ 2 , . .

Original: γ n ) to model seasonality.
Revised: γ n ) to model seasonality.

Original: We use a binary vector z a =
Revised: We use a binary vector z a =

Original: ( z a 1 , z a 2 , . . .
Revised: ( z a 1 , z a 2 , . . .

Original: , z an ) to indicate anomaly points.
Revised: , z an ) to indicate anomaly points.

Original: Then we have
Revised: Then we have

Original: Observation equation:
Revised: Observation equation

Original: [Equation]
Revised: [Equation]

Original: The deviation between the observation y t and its “mean” µ t + γ t is modeled by (cid:15) t and o t , depending on the value of z at .
Revised: The deviation between the observation y t and its “mean” µ t + γ t is modeled by (cid:15) t and o t , depending on the value of z at .

Original: If z at = 1 , then y t is an anomaly point; otherwise it is not.
Revised: If z at = 1 , then y t is an anomaly point; otherwise it is not.

Original: Distinguished from the residues (cid:15) =
Revised: Distinguished from the residues (cid:15) =

Original: ( (cid:15) 1 , (cid:15) 2 , . .
Revised: ( (cid:15) 1 , (cid:15) 2 , . .

Original: , (cid:15) n ) , the anomaly is captured by o =
Revised: , (cid:15) n ) , the anomaly is captured by o =

Original: ( o 1 , o 2 , . . .
Revised: ( o 1 , o 2 , . . .

Original: , o n ) which has relative large magnitude.
Revised: , o n ) which has relative large magnitude.

Original: The hidden state variable µ and γ have intrinsic structures.
Revised: The hidden state variable µ and γ have intrinsic structures.

Original: There are two transition equations, for trend and seasonality separately
Revised: There are two transition equations, for trend and seasonality separately

Original: Transition Equations:
Revised: Transition Equations:

Original: [Equation]
Revised: [Equation]

Original: In Equation (2), δ = ( δ 1 , δ 2 , . . .
Revised: In Equation (2), δ = ( δ 1 , δ 2 , . . .

Original: , δ n ) can be viewed as the “slope” of the trend, measuring how fast the trend changes over time.
Revised: , δ n ) can be viewed as the “slope” of the trend, measuring how fast the trend changes over time.

Original: The change point component is also incorporated in Equation (2) by a binary vector z c = ( z c 1 , z c 2 , . . .
Revised: The change point component is also incorporated in Equation (2) by a binary vector z c = ( z c 1 , z c 2 , . . .

Original: , z cn ) .
Revised: , z cn ) .

Original: If z ct = 1 , it means the t -th point is a change point, with µ t differs from µ t − 1 + δ t − 1 (which can be interpreted as the “momentum” from the previous status ) by r t ; otherwise it is not a change point and they differ by u t .
Revised: If z ct = 1 , it means the t -th point is a change point, with µ t differs from µ t − 1 + δ t − 1 (which can be interpreted as the “momentum” from the previous status ) by r t ; otherwise it is not a change point and they differ by u t .

Original: We model the change points in a way such that r = ( r 1 , r 2 , . . .
Revised: We model the change points in a way such that r = ( r 1 , r 2 , . . .

Original: , r n ) have larger magnitude compared u
Revised: , r n ) have larger magnitude compared u

Original: = ( u 1 , u 2 , . . . , u n ) .
Revised: = ( u 1 , u 2 , . . . , u n ) .

Original: The “slope” part δ also has its own noise v =
Revised: The “slope” part δ also has its own noise v =

Original: ( v 1 , v 2 , . .
Revised: ( v 1 , v 2 , . .

Original: . , v n ) .
Revised: . , v n ) .

Original: A ﬁrst look on Equation (2) may bring up with the question that it is not presented in an exactly the same way as shown in Figure 1.
Revised: A ﬁrst look on Equation (2) may bring up with the question that it is not presented in an exactly the same way as shown in Figure 1.

Original: In Figure 1, the change points component is a step function, and it is one of the ﬁve additive components along with trend, seasonality, anomaly points and residuals.
Revised: In Figure 1, the change points component is a step function, and it is one of the ﬁve additive components along with trend, seasonality, anomaly points and residuals.

Original: Here we model the change point directly into the trend component.
Revised: Here we model the change point directly into the trend component.

Original: Though differing in formulation, they are equivalent to each other.
Revised: Though differing in formulation, they are equivalent to each other.

Original: We choose to model in as in Equation (2) due to simplicity, and its similarity with the deﬁnition of anomaly points in Equation (1).
Revised: We choose to model in as in Equation (2) due to simplicity, and its similarity with the deﬁnition of anomaly points in Equation (1).

Original: The seasonality component is presented in Equation (3).
Revised: The seasonality component is presented in Equation (3).

Original: Here S is the length of one season and w = ( w 1 , w 2 , . . .
Revised: Here S is the length of one season and w = ( w 1 , w 2 , . . .

Original: , w n ) is the noise for seasonality.
Revised: , w n ) is the noise for seasonality.

Original: The seasonality component is assumed to have almost zero average in each season.
Revised: The seasonality component is assumed to have almost zero average in each season.

Original: The observation equation and transition equationstogether (i.e., Equation (1,2,3))together deﬁne how y is generated from all the hidden variables including change points and anomaly points.
Revised: The observation equation and transition equations (i.e., Equation (1,2,3)) deﬁne how y is generated from all the hidden variables including change points and anomaly points.

Original: We continue to explore this new model, under a Bayesian framework.
Revised: We continue to explore this new model, under a Bayesian framework.

Original:  B AYESIAN F RAMEWORK
Revised:  B AYESIAN F RAMEWORK

Original: Bayesian methods are widely used in many data analysis ﬁelds.
Revised: Bayesian methods are widely used in many data analysis ﬁelds.

Original: It is easy to implement and interpret, and it also has the ability to produce posterior distribution.
Revised: It is easy to implement and interpret, and it also has the ability to produce posterior distribution.

Original: The Bayesian method on state space time series model has been investigated in Scott & Varian (2014); Brodersen et al.
Revised: The Bayesian method on state space time series model has been investigated in Scott & Varian (2014); Brodersen et al.

Original: In this section, we also consider Bayesian framework for our novel state space time series model.
Revised: In this section, we also consider Bayesian framework for our novel state space time series model.

Original: We assume all the noises are normally distributed
Revised: We assume all the noises are normally distributed

Original: [Equation]
Revised: [Equation]

Original: where σ (cid:15) , σ o , σ u , σ r , σ v , σ w are parameters for standard deviation.
Revised: where σ (cid:15) , σ o , σ u , σ r , σ v , σ w are parameters for standard deviation.

Original: As binary vectors, a natural choice is to model anomaly point indicator
Revised: As binary vectors, a natural choice is to model anomaly point indicator

Original: z a and change point indicator z c to the model them as
Revised: z a and change point indicator z c to the model them as

Original: [Table]
Revised: Bernoulli random variables

Original: [Equation]
Revised: [Equation]

Original: where p a , p c are probabilities for each point to be an anomaly or change point.
Revised: where p a , p c are probabilities for each point to be an anomaly or change point.

Original: For simplicity, we denote α t =
Revised: For simplicity, we denote α t =

Original: ( µ t , δ t , γ t , γ t − 1 , . . .
Revised: ( µ t , δ t , γ t , γ t − 1 , . . .

Original: , γ t − ( S − 2) ) to include the main hidden variables (except z at and z ct ) in the transition equations.
Revised: , γ t − ( S − 2) ) to include the main hidden variables (except z at and z ct ) in the transition equations.

Original: All the α t are well deﬁned and can be generated from the previous status, except α 1 .
Revised: All the α t are well deﬁned and can be generated from the previous status, except α 1 .

Original: We denote a 1 to be the parameter for α 1 , which can be interpreted as the “mean” for α 1 .
Revised: We denote a 1 to be the parameter for α 1 , which can be interpreted as the “mean” for α 1 .

Original: With Bayesian framework, we are able to represent our model graphically as in Figure 2.
Revised: With Bayesian framework, we are able to represent our model graphically as in Figure 2.

Original: As shown in Figure 2, the only observations are y and all the others are hidden.
Revised: As shown in Figure 2, the only observations are y and all the others are hidden.

Original: In this paper, we assume there is no additional information on all the hidden states.
Revised: In this paper, we assume there is no additional information on all the hidden states.

Original: If we have some prior information, for example, some points are more likely to be change points, then our model can be easily modiﬁed to incorporate such information, by using proper prior.
Revised: If we have some prior information, for example, some points are more likely to be change points, then our model can be easily modiﬁed to incorporate such information, by using proper prior.

Original: In Figure 2, we use squares and circles to classify unknown variables.
Revised: In Figure 2, we use squares and circles to classify unknown variables.

Original: Despite all being unknown, they actually behave differently according to their own functionality.
Revised: Despite all being unknown, they actually behave differently according to their own functionality.

Original: For those in squares, they behave like turning parameters.
Revised: For those in squares, they behave like turning parameters.

Original: Once they are initialized or given, those in circles behaves like latent variables.
Revised: Once they are initialized or given, those in circles behaves like latent variables.

Original: We call the former “parameters” and the latter “latent variable”, as listed in Table 1.
Revised: We call the former “parameters” and the latter “latent variable”, as listed in Table 1.

Original: [Table]
Revised: [Table]

Original: The discrepancy between these two categories is clearly captured by the joint likelihood function. 
Revised: The discrepancy between these two categories is clearly captured by the joint likelihood function. From Figure

Original: From Figure 2, the joint distribution (i.e., the likelihood function can be written down explicitly as
Revised: [Table] the joint distribution (i [Table] e., the likelihood function) can be written down explicitly as

Original: L a 1 , p , σ ( y , α , z ) (
Revised: L a 1 , p , σ ( y , α , z ) (4) =

Revised: (cid:89) { t : z at =0 } g

Revised: ( y t − µ t − γ t ,σ (cid:15) )

Revised: × (cid:89) { t : z at =1 } g ( y t − µ

Original: [Equation]
Revised: [Table] − [Table]

Original: = (cid:89) { t : z at =0 } [Equation]
Revised: σ o ) [Table] (cid:89) { t : z ct =0 } g

Original: ×
Revised: [Table] − [Table] − δ t −

Original: [Equation]
Revised: [Table] σ u [Table]

Revised: × (cid:89) { t : z ct =1 } g

Original: [Equation]
Revised: [Table]

Revised: µ t − µ

Original: where g ( x 1 , x 2 )=
Revised: [Table] − δ t − [Table] σ r )

Original:  1 √ 2 πx 2 exp (cid:0) − x 21 / (2 x 22 ) (cid:1) is the density function for normal distribution with mean x 1 and standard deviation x 2 .
Revised: [Table] n [Table] (1 − p a ) 1 − z at ( p c ) z ct (1 − p c ) 1 − z ct , where g ( x 1 , x 2 ) = 1 √ 2 πx 2 exp (cid:0) − x 21 / (2 x 22 ) (cid:1) is the density function for normal distribution with mean x 1 and standard deviation x 2 .

Original: Here we slightly abuse the notation by using µ 0 , δ 0 , γ 0 , γ − 1 , . .
Revised: Here we slightly abuse the notation by using µ 0 , δ 0 , γ 0 , γ − 1 , . .

Original: , γ 2 − S , which are actually the corresponding coordinates of a 1 .
Revised: , γ 2 − S , which are actually the corresponding coordinates of a 1 .

Original: As long with other probabilistic graphical models, our model can also be viewed as a generative model.
Revised: As long with other probabilistic graphical models, our model can also be viewed as a generative model.

Original: Given the parameters a 1 , p , σ , we are able to generate time series.
Revised: Given the parameters a 1 , p , σ , we are able to generate time series.

Original: We present the generative procedure as follows.
Revised: We present the generative procedure as follows.

Original: Input: Parameters a 1 , σ = ( σ (cid:15) , σ o , σ u , σ r , σ v , σ w ) and p a , p c , length of time series to generate m
Revised: Input: Parameters a 1 , σ = ( σ (cid:15) , σ o , σ u , σ r , σ v , σ w ) and p a , p c , length of time series to generate m

Original: Output: Time series y =
Revised: Output: Time series y =

Original: ( y 1 , y 2 , . . .
Revised: ( y 1 , y 2 , . . .

Original: , y m ) 1 Generate the indexes where anomalies or change points occur 
Revised: , y m ) 1 Generate the indexes where anomalies or change points occur { z at } nt =

Original: [Equation] ; 2 Generate all the noises (cid:15) , o , u , r , v , w as independent normal random variables with mean zero and standard deviation σ (cid:15) , σ o , σ u , σ r , σ v , σ w respectively; 3 Generate { α t } mt =1 sequentially by the transition functions in Equation (2) and (3); 4 Generate time series { y t } mt =1 by the observation function in Equation (1).
Revised: 1 iid ∼ Ber ( p a ) , { z ct } nt =1 iid ∼ Ber ( p c ); 2 Generate all the noises (cid:15) , o , u , r , v , w as independent normal random variables with mean zero and standard deviation σ (cid:15) , σ o , σ u , σ r , σ v , σ w respectively; 3 Generate { α t } mt =1 sequentially by the transition functions in Equation (2) and (3); 4 Generate time series { y t } mt =1 by the observation function in Equation (1).

Original:  I NFERENCE
Revised:  I NFERENCE

Original: This section is about inferring unknown variables from y , given the Bayesian setting described in the previous section.
Revised: This section is about inferring unknown variables from y , given the Bayesian setting described in the previous section.

Original: The main framework here is to sequentially update each hidden variable by ﬁxing the remaining ones.
Revised: The main framework here is to sequentially update each hidden variable by ﬁxing the remaining ones.

Original: As stated in the previous section, there are two different categories of unknown variables.
Revised: As stated in the previous section, there are two different categories of unknown variables.

Original: Different update schemes need to be used due to the difference in their functionality.
Revised: Different update schemes need to be used due to the difference in their functionality.

Original: For the latent variables, we implement Markov chain Monte Carlo (MCMC) for inference.
Revised: For the latent variables, we implement Markov chain Monte Carlo (MCMC) for inference.

Original: Particular, we use Gibbs sampler.
Revised: Particular, we use Gibbs sampler.

Original: We will elaborate the details of updates in the following sections.
Revised: We will elaborate the details of updates in the following sections.

Original: 4. U PDATES ON T REND AND S EASONALITY
Revised: 4. U PDATES ON T REND AND S EASONALITY

Original: In this section, we focus on updating α assuming all the other hidden variables are given and ﬁxed.
Revised: In this section, we focus on updating α assuming all the other hidden variables are given and ﬁxed.

Original: The essence of Gibbs sampler is to obtain posterior distribution p a 1 , p , σ ( α |
Revised: The essence of Gibbs sampler is to obtain posterior distribution p a 1 , p , σ ( α |

Original: y , z ) .
Revised: y , z ) .

Original: This can be achieved by a combination of Kalman ﬁlter, Kalman smoothing and the so-called “fake-path” trick.
Revised: This can be achieved by a combination of Kalman ﬁlter, Kalman smoothing and the so-called “fake-path” trick.

Original: We provide some intuitive explanation here and refer the readers to Durbin & Koopman (2012) for detailed implementation.
Revised: We provide some intuitive explanation here and refer the readers to Durbin & Koopman (2012) for detailed implementation.

Original: Kalman ﬁlter and Kalman smoothing are classic algorithms in signal processing and pattern recolonization for Bayesian inference.
Revised: Kalman ﬁlter and Kalman smoothing are classic algorithms in signal processing and pattern recolonization for Bayesian inference.

Original: It is well related to other algorithms especially message passing algorithm.
Revised: It is well related to other algorithms especially message passing algorithm.

Original: Kalman ﬁlter collects information forwards to obtain E ( α t | y 1 , y 2 , . .
Revised: Kalman ﬁlter collects information forwards to obtain E ( α t | y 1 , y 2 , . .

Original: . , y t ) ; while Kalman smoothing distribute information backwards to achieve E ( α t | y ) .
Revised: . , y t ) ; while Kalman smoothing distribute information backwards to achieve E ( α t | y ) .

Original: However, the combination of Kalman ﬁlter and Kalman smoothing is not enough, as it only gives the the expectations of marginal distributions { E ( α t | y ) } nt =1 , instead of the joint distribution required for Gibbs sampler.
Revised: However, the combination of Kalman ﬁlter and Kalman smoothing is not enough, as it only gives the the expectations of marginal distributions { E ( α t | y ) } nt =1 , instead of the joint distribution required for Gibbs sampler.

Original: To address this issue, we can use the “fake-path” trick described in Brodersen et al.
Revised: To address this issue, we can use the “fake-path” trick described in Brodersen et al.

Original: ; Durbin & Koopman (2012).
Revised: ; Durbin & Koopman (2012).

Original: The main idea underlying this trick lies on the fact that the covariance structure of p ( α t | y ) is not dependent on the means.
Revised: The main idea underlying this trick lies on the fact that the covariance structure of p ( α t | y ) is not dependent on the means.

Original: If we are able to obtain the covariance by some other way, then we can add it up with { E ( α t | y ) } nt =1 to obtain a sample from p ( α | y ) .
Revised: If we are able to obtain the covariance by some other way, then we can add it up with { E ( α t | y ) } n t =1 to obtain a sample from p ( α | y ) .

Original: This trick involves three steps.
Revised: This trick involves three steps.

Original: Note that all the other hidden variables z , p , σ are given.
Revised: Note that all the other hidden variables z , p , σ are given.

Original: Pick some vector ˜ a 1 , and generate a sequence of time series ˜ y from it by Algorithm 1.
Revised: Pick some vector ˜ a 1 , and generate a sequence of time series ˜ y from it by Algorithm 1.

Original: In this way, we also observe ˜ α . 2.
Revised: In this way, we also observe ˜ α . 2.

Original: Obtain { E ( ˜ α t | ˜ y ) } nt =1 from ˜ y by Kalman ﬁlter and Kalman smoothing.
Revised: Obtain { E ( ˜ α t | ˜ y ) } nt =1 from ˜ y by Kalman ﬁlter and Kalman smoothing.

Original: We use { ˜ α t − E ( ˜ α t | ˜ y )
Revised: We use { ˜ α t − E ( ˜ α t | ˜ y )

Original: + E ( α t | y ) } nt 
Revised: + E ( α t | y ) } nt =

Original: =1 as our sampling from the conditional distribution.
Revised: 1 as our sampling from the conditional distribution.

Original: 4. C HANGE P OINT AND A NOMALY D ETECTION
Revised: 4. C HANGE P OINT AND A NOMALY D ETECTION

Original: In this section, we update z by Gibbs sampler, assuming α , a 1 , p , σ are all given and ﬁxed.
Revised: In this section, we update z by Gibbs sampler, assuming α , a 1 , p , σ are all given and ﬁxed.

Original: We need to obtain the conditional distribution p a 1 , p , σ ( z | y , α ) .
Revised: We need to obtain the conditional distribution p a 1 , p , σ ( z | y , α ) .

Original: Note that in the graphical model described in
Revised: Note that in the graphical model described in

Original: Section 2, { z at } nt =1 and { z ct } ni =1 are all Bernoulli random variables and independent of each other.
Revised: Section 2, { z at } nt =1 and { z ct } ni =1 are all Bernoulli random variables and independent of each other.

Original: Then the conditional distribution p a 1 , p , σ ( z | y , α ) can also be decomposed into product of Bernoulli density functions.
Revised: Then the conditional distribution p a 1 , p , σ ( z | y , α ) can also be decomposed into product of Bernoulli density functions.

Original: In other words, conditioned on y , α , { z at } nt =1 and { z ct } ni =1 are still independent
Revised: In other words, conditioned on y , α , { z at } nt =1 and { z ct } ni =1 are still independent

Original: Bernoulli random variables, but possibly with different success probabilities.
Revised: Bernoulli random variables, but possibly with different success probabilities.

Original: Thus, we can take the calculation point by point.
Revised: Thus, we can take the calculation point by point.

Original: For example, for the anomaly detection for the t -th point, we have
Revised: For example, for the anomaly detection for the t -th point, we have

Original: [Equation]
Revised: [Equation]

Original: And the prior on z at is P ( z at = 1) =
Revised: And the prior on z at is P ( z at = 1) =

Original: p a and P ( z at = 0) =
Revised: p a and P ( z at = 0) =

Original: p 1 .
Revised: p 1 .

Original: Let p at = P
Revised: Let p at = P

Original: ( z at = 1 | y , α ) .
Revised: ( z at = 1 | y , α ) .

Original: Directly calculation leads to
Revised: Directly calculation leads to

Original: [Equation]
Revised: [Equation]

Original: This equality holds for all t = 1 , 2 , . . . , n .
Revised: This equality holds for all t = 1 , 2 , . . . , n .

Original: Similarly for change point detection, let p ct
Revised: Similarly for change point detection, let p ct

Original: = P ( z ct = 1 |
Revised: = P ( z ct = 1 |

Original: y , α ) , and we have
Revised: y , α ) , and we have

Original: [Equation]
Revised: [Equation]

Original: As mentioned above, all the coordinates in z are still independent Bernoulli random variables conditioned on y , α .
Revised: As mentioned above, all the coordinates in z are still independent Bernoulli random variables conditioned on y , α .

Original: Thus, for Gibbs sampler, we can generate z by sampling independently with [Equation]
Revised: Thus, for Gibbs sampler, we can generate z by sampling independently with { z at } nt =1 ∼ Ber ( p at ) , { z ct } nt =1 ∼ Ber ( p ct ) .

Original: For change point detection here, we have an additional segment control step.
Revised: For change point detection here, we have an additional segment control step.

Original: After obtaining { z ct } nt =1 as mentioned above, we need to make sure that the change points detected satisfy some additional requirement on the length of segment among two consecutive change points.
Revised: After obtaining { z ct } nt =1 as mentioned above, we need to make sure that the change points detected satisfy some additional requirement on the length of segment among two consecutive change points.

Original: This issue arises from the ambiguity between the deﬁnitions of change point and anomaly points.
Revised: This issue arises from the ambiguity between the deﬁnitions of change point and anomaly points.

Original: For example, consider a time series with value (0 , 0 , 0 , 0 , 1 , 1 , 1 , 0 , 0 , 0) .
Revised: For example, consider a time series with value (0 , 0 , 0 , 0 , 1 , 1 , 1 , 0 , 0 , 0) .

Original: We can view it with two change points, one increases the trend by 1 and the other decreases it by 1.
Revised: We can view it with two change points, one increases the trend by 1 and the other decreases it by 1.

Original: Alternatively, we can also argue the three 1s in this time series are anomalies, though next to each other.
Revised: Alternatively, we can also argue the three 1s in this time series are anomalies, though next to each other.

Original: One way to address this ambiguity is by deﬁning the minimum length of segment (denoted as l ).
Revised: One way to address this ambiguity is by deﬁning the minimum length of segment (denoted as (cid:96) ).

Original: In this toy example, if we set the minimum length to be 4, then they are anomaly points; if we set it to be 3, then we regard them to be change points.
Revised: In this toy example, if we set the minimum length to be 4, then they are anomaly points; if we set it to be 3, then we regard them to be change points.

Original: But a more complicated criterion is needed than using minimum length as the time series usually own much more complex structure than this toy example.
Revised: But a more complicated criterion is needed than using minimum length as the time series usually own much more complex structure than this toy example.

Original: Consider time series (0 , 0 , 0 , 0 , − 1 , − 1 , 1 , 1 , 1 , 1) and the minimum time series parameter l = 3 .
Revised: Consider time series (0 , 0 , 0 , 0 , − 1 , − 1 , 1 , 1 , 1 , 1) and the minimum time series parameter (cid:96) = 3 .

Original: It is reasonable to view it with one change point with increment 1, and the two -1s should be regarded as anomalies.
Revised: It is reasonable to view it with one change point with increment 1, and the two -1s should be regarded as anomalies.

Original: As a combination of all these factors, we propose the following segment control method.
Revised: As a combination of all these factors, we propose the following segment control method.

Original: A default value for the parameter l is the length of seasonality, i.e., l = S . 
Revised: A default value for the parameter (cid:96) is the length of seasonality, i.e., (cid:96) = S . Algorithm

Original:  while there exists i such that | t i +1 − t i | < l do
Revised: Input minimum segment (cid:96) Output: change point binary vector z c 1 Denote t 1 < t 2 < . . . to be all the indexes such that z c t i = 1 ; while there exists i such that | t i +1 − t i | < (cid:96) do

Original:  − µ 
Revised: Check if | µ t i − 1 − µ t

Original: 1 +1 | ≤ σ r / 2 .
Revised: i +1 +1 | ≤ σ r / 2 .

Original: If so, exclude both them from change points by setting z ct i
Revised: If so, exclude both them from change points by setting z ct i

Original: =
Revised: =

Original: z ct i +

Revised: z ct i +

Original: = 0 .
Revised: = 0 .

Original: Otherwise, randomly exclude one of them by setting the corresponding coordinate in z c to be 0; Update all the indexes of change points in z c . 
Revised: Otherwise, randomly exclude one of them by setting the corresponding coordinate in z c to be 0; Update all the indexes of change points in z c . end

Original: 4. I NITIALIZATION AND U PDATES ON P ARAMETERS
Revised: 4. I NITIALIZATION AND U PDATES ON P ARAMETERS

Original: The parameters σ , a 1 and p need both initialization and update.
Revised: The parameters σ , a 1 and p need both initialization and update.

Original: We have different initializations and update schemes for each of them.
Revised: We have different initializations and update schemes for each of them.

Original: For all the standard deviations, once we obtain α and z , we update them by taking the empirical standard deviation correspondingly.
Revised: For all the standard deviations, once we obtain α and z , we update them by taking the empirical standard deviation correspondingly.

Original: For σ δ and σ γ , the calculation is straightforward as they only involve δ and γ respectively.
Revised: For σ δ and σ γ , the calculation is straightforward as they only involve δ and γ respectively.

Original: For σ (cid:15) , σ o , σ u and σ r , it is a bit more involved due to z .
Revised: For σ (cid:15) , σ o , σ u and σ r , it is a bit more involved due to z .

Original: Nevertheless, we can obtain the following update equations for all of them:
Revised: Nevertheless, we can obtain the following update equations for all of them:

Original: [Equation]
Revised: [Equation]

Original: Note that in some iterations, when there is no change point or anomaly detected in z , then the updates above for σ o , σ r are not well-deﬁned.
Revised: Note that in some iterations, when there is no change point or anomaly detected in z , then the updates above for σ o , σ r are not well-deﬁned.

Original: In those cases, we simply let them remain the same.
Revised: In those cases, we simply let them remain the same.

Original: To initialize σ , we let them all equal to the standard deviation of y .
Revised: To initialize σ , we let them all equal to the standard deviation of y .

Original: For a 1 , we initialize it by letting its ﬁrst coordinate to be equal to the average of y 1 , y 2 , . . .
Revised: For a 1 , we initialize it by letting its ﬁrst coordinate to be equal to the average of y 1 , y 2 , . . .

Original: , y S , and all the remaining coordinates to be equal to 0.
Revised: , y S , and all the remaining coordinates to be equal to 0.

Original: Since a 1 can be interpreted as the mean vector of α 1 , in this way the trend is initialized to be matched up with average of the ﬁrst season, and the slope and seasonality are initialized to be equal to 0.
Revised: Since a 1 can be interpreted as the mean vector of α 1 , in this way the trend is initialized to be matched up with average of the ﬁrst season, and the slope and seasonality are initialized to be equal to 0.

Original: We update a 1 by using information of α .
Revised: We update a 1 by using information of α .

Original: We let the ﬁrst two coordinates (trend and slope) of a 1 to be equal to those of α 1 , and we let the remaining coordinates (seasonality) of a 1 to be equal to those of α S
Revised: We let the ﬁrst two coordinates (trend and slope) of a 1 to be equal to those of α 1 , and we let the remaining coordinates (seasonality) of a 1 to be equal to those of α S

Original: +1 .
Revised: +1 .

Original: The reason why we do not let a 1 to be equal to α 1 entirely is due to the consideration on convergence and robustness.
Revised: The reason why we do not let a 1 to be equal to α 1 entirely is due to the consideration on convergence and robustness.

Original: Since we initialize the seasonality part in a 1 as 0, it will remain 0 if we let a 1 equals α 1 entirely (due to the mechanism how we update α 1 as described in Section 4.1.
Revised: Since we initialize the seasonality part in a 1 as 0, it will remain 0 if we let a 1 equals α 1 entirely (due to the mechanism how we update α 1 as described in Section 4.1.

Original: We can avoid such trouble via using α S +1 .
Revised: We can avoid such trouble via using α S +1 .

Original: For p , we initialize them to be equal to 1 /n .
Revised: For p , we initialize them to be equal to 1 /n .

Original: If we have additional information on the number of change points or anomaly points, we can initiate them with different values, for example, 0 .
Revised: If we have additional information on the number of change points or anomaly points, we can initiate them with different values, for example, 0 .

Original: 1 /n , or 10 /n .
Revised: 1 /n , or 10 /n .

Original: We can update p after obtaining z , but we choose not to, also for the sake of robustness.
Revised: We can update p after obtaining z , but we choose not to, also for the sake of robustness.

Original: In the early iterations when the algorithm is far from convergence, it is highly possible that z a or z c may turn out to be all 0.
Revised: In the early iterations when the algorithm is far from convergence, it is highly possible that z a or z c may turn out to be all 0.

Original: If we update p , say, by taking the proportion of change point or anomaly points in z .
Revised: If we update p , say, by taking the proportion of change point or anomaly points in z .

Original: Then p a or p c might be 0, and it may get stuck in 0 in the remaining iterations.
Revised: Then p a or p c might be 0, and it may get stuck in 0 in the remaining iterations.

Original:  F ORECASTING
Revised:  F ORECASTING

Original: Once we infer all the latent variables α , z and tune all the parameters p , a 1 , σ , we are able to forecast the future time series y future .
Revised: Once we infer all the latent variables α , z and tune all the parameters p , a 1 , σ , we are able to forecast the future time series y future .

Original: From the graphical model described in Section 3, the future forecasting only involves α n instead of the whole α .
Revised: From the graphical model described in Section 3, the future forecasting only involves α n instead of the whole α .

Original: Note that we assume that there exists no change point and anomaly point in the future.
Revised: Note that we assume that there exists no change point and anomaly point in the future.

Original: This is reasonable as in most cases we have no additional information on the future time series.
Revised: This is reasonable as in most cases we have no additional information on the future time series.

Original: Given α n and σ we can use our predictive procedures (i.e.,
Revised: Given α n and σ we can use our predictive procedures (i.e.,

Original: Algorithm 1) to generate future time series y future .
Revised: Algorithm 1) to generate future time series y future .

Original: We can further integral out α n to have the posterior predictive distribution as p σ ( y future | y ) .
Revised: We can further integrate out α n to have the posterior predictive distribution as p σ ( y future | y ) .

Original: The forecasting on future time series is not deterministic.
Revised: The forecasting on future time series is not deterministic.

Original: There are two sources for the randomness in y future .
Revised: There are two sources for the randomness in y future .

Original: One comes from the inference of α n (and also σ ) from y .
Revised: One comes from the inference of α n (and also σ ) from y .

Original: Under the Bayesian framework in Section 3, we have a posterior distribution over α n rather than a single point estimation.
Revised: Under the Bayesian framework in Section 3, we have a posterior distribution over α n rather than a single point estimation.

Original: The second one comes from the forecasting function itself.
Revised: The second one comes from the forecasting function itself.

Original: The forecasting involves intrinsic noise like (cid:15) t , u t , v t and w t .
Revised: The forecasting involves intrinsic noise like (cid:15) t , u t , v t and w t .

Original: Thus, the predictive density function p σ ( y future | y , α n ) will lead to different path even with ﬁxed σ and α n .
Revised: Thus, the predictive density function p σ ( y future | y , α n ) will lead to different path even with ﬁxed σ and α n .

Original: In this way we are able to obtain distribution and predictive interval for forecasting.
Revised: In this way we are able to obtain distribution and predictive interval for forecasting.

Original: We also suggest to take the average of multiple forecasting paths, as the posterior mean for the forecasting.
Revised: We also suggest to take the average of multiple forecasting paths, as the posterior mean for the forecasting.

Original: The average of multiple forecasting paths (denoted as ¯ y future ), if the number of paths is large enough, always takes the form as a combination of linear trend and seasonality.
Revised: The average of multiple forecasting paths (denoted as ¯ y future ), if the number of paths is large enough, always takes the form as a combination of linear trend and seasonality.

Original: This can be observed in both our synthesis data (Section 7) and real data analysis (Section 8).
Revised: This can be observed in both our synthesis data (Section 7) and real data analysis (Section 8).

Original: This seems to be surprising at the ﬁrst glance, but makes some sense intuitively.
Revised: This seems to be surprising at the ﬁrst glance, but makes some sense intuitively.

Original: Under our assumption, we have no information on the future, and thus a reliable way to forecast the future is to use the information collected at the end of observed time series, i.e., trend µ n , slope δ n and seasonality structure.
Revised: Under our assumption, we have no information on the future, and thus a reliable way to forecast the future is to use the information collected at the end of observed time series, i.e., trend µ n , slope δ n and seasonality structure.

Original: Theorem 1 gives mathematical explanation of the linearity of ¯ y future , in both mean and standard deviation.
Revised: Theorem 1 gives mathematical explanation of the linearity of ¯ y future , in both mean and standard deviation.

Original: Theorem 1.
Revised: Theorem 1.

Original: Let N be the number of future time series paths we generate from Algorithm 1).
Revised: Let N be the number of future time series paths we generate from Algorithm 1).

Original: Let m be the number of points we are going to forecast.
Revised: Let m be the number of points we are going to forecast.

Original: Denote { y (1) n + j } m j =1 , { y (2) n + j } m j =1 , . .
Revised: Denote { y (1) n + j } mj =1 , { y (2) n + j } mj =1 , . .

Original: , { y ( N ) n + j } m j = to be the future paths.
Revised: , { y ( N ) n + j } mj = to be the future paths.

Original: Deﬁne ¯ y future
Revised: Deﬁne ¯ y future

Original: =
Revised: =

Original: (¯ y n +1 , ¯ y n +2 , . .
Revised: (¯ y n +1 , ¯ y n +2 , . .

Original: , ¯ y n + m ) to be the average such that
Revised: , ¯ y n + m ) to be the average such that

Original: [Equation]
Revised: [Equation]

Original: Then for all j = 1 , 2 , . .
Revised: Then for all j = 1 , 2 , . .

Original: , N , we have ¯ y n + j as a normal distribution with mean and variance as
Revised: , N , we have ¯ y n + j as a normal distribution with mean and variance as

Original: [Equation]
Revised: [Equation]

Original: Consequently, for all j = 1 , 2 , . .
Revised: Consequently, for all j = 1 , 2 , . .

Original: , m , E [¯ y n + j ] is in a linear form with respect to j , and the standard deviation of ¯ y n + j also takes a approximately linear form with respect to j .
Revised: , m , E [¯ y n + j ] is in a linear form with respect to j , and the standard deviation of ¯ y n + j also takes a approximately linear form with respect to j .

Original: Proof.
Revised: Proof.

Original: Recall that α n , σ are given and ﬁxed, and we assume there is no change point or anomaly in the future time series.
Revised: Recall that α n , σ are given and ﬁxed, and we assume there is no change point or anomaly in the future time series.

Original: The Equation (2) leads to δ n + j = δ n + (cid:80) jl 
Revised: The Equation (2) leads to δ n + j = δ n + (cid:80) jl =

Original: [Equation]

Original: 1 v n + l , which implies that
Revised: 1 v n + l , which implies that

Original: [Equation]
Revised: [Equation]

Original: For the seasonality part, simple linear algebra together with Equation 3 leads to γ n + j = γ n − S +( j mod S )
Revised: For the seasonality part, simple linear algebra together with Equation 3 leads to γ n + j = γ n − S +( j mod S )

Original: + (cid:80) jl = 1 w n + l .
Revised: + (cid:80) j l =1 w n + l .

Original: Thus,¯
Revised: Thus,

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: Due to the independence and Gaussian distribution of all the noises, ¯ y n + j is also normally distributed and its means and variance can be calculated accordingly.
Revised: Due to the independence and Gaussian distribution of all the noises, ¯ y n + j is also normally distributed and its means and variance can be calculated accordingly.

Original:  A LGORITHM
Revised:  A LGORITHM

Original: Our proposed method can be divided into three parts: initialization, inference, and forecasting.
Revised: Our proposed method can be divided into three parts: initialization, inference, and forecasting.

Original: Section 4 and Section 5 provide detailed explanation and reasoning for each of them.
Revised: Section 4 and Section 5 provide detailed explanation and reasoning for each of them.

Original: We present a whole picture of our proposed methodology in Algorithm 3. 
Revised: We present a whole picture of our proposed methodology in Algorithm 3. Algorithm Proposed Algorithm

Revised: Part

Original: It is worth mentioning that our proposed methodology is downward compatible with many simpler state space time series models.
Revised: I Initialization ; 1 Initialize σ (cid:15) , σ o , σ u , σ r , σ v , σ w all with the empirical standard deviation of y ; 2 Initialize a 1 such that its ﬁrst coordinate equals to the average of ( y 1 , y 2 , . . . , y S ) and all the remaining S coordinates with 0; 3 Initialize p a and p c by 1 /n .

Revised: Then generate z a and z c as independent Bernoulli random variables with success probability p a and p c respectively;

Revised: Part II: Inference ; while the likelihood function L a 1 , p , σ ( y , α , z ) not converges do

Revised: Infer α by Kalman ﬁlter, Kalman smoothing and “fake-path” trick described in Section 4.1;

Revised: Update z

Revised: a and

Revised: z

Revised: c by sampling from

Revised: [Equation]

Revised: , where the success probability { p at } nt =1 and { p ct } nt =1 are deﬁned in Equation (5) and (6)

Revised: Segment control on

Revised: z c

Revised: by

Revised: Algorithm

Revised: ; Update σ by Equation (7) to (8); Update a 1 such that its ﬁrst two coordinates equal to the those of α 1 and the remaining ( S − 1) coordinates equals to those of α S +1 ; Calculate the likelihood function L a 1 , p , σ ( y , α , z ) given in Equation (4); end Part III: Forecasting ;

Revised: a n and σ , use the generate procedure in Algorithm 1 to generate future time series y future with length m .

Revised: Repeat the generative procedure to obtain multiple future paths y (1)

Original: By letting p c = 0 , we assume there is no change point in the  time series.
Revised: , y (2) , y ( all the predictive paths give the distribution for the future time series forecasting. If needed, calculate the point-wise quantile to obtain predictive intervals.

Original:  By letting p a = 0 , we assume there is no anomaly point in the time series.
Revised: the point-wise average as our ﬁnal forecasting result. It is worth mentioning that our proposed methodology is downward compatible with many simpler state space time series models. By letting p c = 0 , we assume there is no change point in the time series. By letting p a = 0 , we assume there is no anomaly point in the time series.

Original: If both p c and p a are set to be 0, then our model is reduced to the classic state space time series model.
Revised: If both p c and p a are set to be 0, then our model is reduced to the classic state space time series model.

Original: Also, the seasonality and slope can be removed from our model, if we know there exists no such structure in the data.
Revised: Also, the seasonality and slope can be removed from our model, if we know there exists no such structure in the data.

Original:  S IMULATION
Revised:  S IMULATION

Original: In this section, we study the synthetic data generated from our model.
Revised: In this section, we study the synthetic data generated from our model.

Original: We let S = 7 and provide values for σ and a 1 .
Revised: We let S = 7 and provide values for σ and a 1 .

Original: The change points and anomaly points are randomly generated.
Revised: The change points and anomaly points are randomly generated.

Original: We use our generative procedure (Algorithm 1) to generate time series with total length 500 by ﬁxed parameters.
Revised: We use our generative procedure (Algorithm 1) to generate time series with total length 500 by ﬁxed parameters.

Original: The ﬁrst 350 points will be used as training set and the remaining 150 points will be used to evaluate the performance of forecasting.
Revised: The ﬁrst 350 points will be used as training set and the remaining 150 points will be used to evaluate the performance of forecasting.

Original: When generating, we let the time series have weekly seasonality with S = 7 .
Revised: When generating, we let the time series have weekly seasonality with S = 7 .

Original: For σ we have σ (cid:15) = 0 .
Revised: For σ we have σ (cid:15) = 0 .

Original: 1 , σ u = 0 .
Revised: 1 , σ u = 0 .

Original: 1 , σ v = 0 .
Revised: 1 , σ v = 0 .

Original: 0004 , σ w = 0 .
Revised: 0004 , σ w = 0 .

Original: 01 , σ r = 1 , σ o = 4 .
Revised: 01 , σ r = 1 , σ o = 4 .

Original: For α 1 we have value for µ as 20, value for δ as 0, and value for seasonality as (1 , 2 , 4 , − 1 , − 3 , − 2) / 10 .
Revised: For α 1 we have value for µ as 20, value for δ as 0, and value for seasonality as (1 , 2 , 4 , − 1 , − 3 , − 2) / 10 .

Original: For p we have p c = 4 /and p a = 10 / 350 .
Revised: For p we have p c = 4 /and p a = 10 / 350 .

Original: Despite that, to make sure that at least one change point is in existence, we force z c 330 = 1 and r 330 = 2 .
Revised: Despite that, to make sure that at least one change point is in existence, we force z c 330 = 1 and r 330 = 2 .

Original: That is, for each time series we generate, its 330th point is a change point with the mean shifted up by 3.
Revised: That is, for each time series we generate, its 330th point is a change point with the mean shifted up by 3.

Original: Also to be consistence with our assumption, we force z c i =
Revised: Also to be consistence with our assumption, we force z ci =

Original: z a i = 0 , ∀ 351 ≤ i ≤ 500 so there exists no change point or anomaly point in the testing part.ﬁrst y y
Revised: z ai = 0 , ∀ 351 ≤ i ≤ 500 so there exists no change point or anomaly point in the testing part.

Original: y S c n z a and z c as independent Bernoulli random variables with success probability p a and p c respectively;

Original: Part II: Inference ; while the likelihood function L a 1 , p , σ ( y , α , z ) not converges do

Original: Infer α by Kalman ﬁlter, Kalman smoothing and “fake-path” trick described in Section 4.1;

Original: Update z a and z c by sampling from

Original: [Equation]

Original: where the success probability { p at } nt =1 and { p ct } nt =1 are deﬁned in Equation (5) and (6);

Original: Segment control on z c by Algorithm 2;

Original: Update σ by Equation (7) to (8);

Original: Update a 1 such that its ﬁrst two coordinates equal to the those of α 1 and the remaining ( S − 1) coordinates equals to those of α S +1 ;

Original: Calculate the likelihood function L a 1 , p , σ ( y , α , z ) given in Equation (4); end

Original: Part

Original: III

Original: Forecasting

Original: future time series length m the generative procedure to obtain multiple future

Original: paths y (1) future , y (2) future , . .

Original: , y ( N ) future ; 11 Combine all the predictive paths give the distribution for the future time series forecasting.

Original: If needed, calculate the point-wise quantile to obtain predictive intervals.

Original: Use the point-wise average as our ﬁnal forecasting result

Original: The top panel of Figure 3 shows one example of synthesis data.
Revised: The top panel of Figure 3 shows one example of synthesis data.

Original: The blue line marks the separation between training and testing set.
Revised: The blue line marks the separation between training and testing set.

Original: The blue dashed line indicates the locations for the change point, while the yellow dots indicate the positions of anomaly points.
Revised: The blue dashed line indicates the locations for the change point, while the yellow dots indicate the positions of anomaly points.

Original: Also see Figure 3 for illustration on the results returned by implementing our proposed algorithm on the same dataset.
Revised: Also see Figure 3 for illustration on the results returned by implementing our proposed algorithm on the same dataset.

Original: The red line gives the ﬁtting results in the ﬁrst 350 points and forecasting results in the last 150 points.
Revised: The red line gives the ﬁtting results in the ﬁrst 350 points and forecasting results in the last 150 points.

Original: The change points detected are marked with vertical red dotted line, and the anomaly detected are ﬂagged with purple squares.
Revised: The change points detected are marked with vertical red dotted line, and the anomaly detected are ﬂagged with purple squares.

Original: Figure 3 shows that on this dataset, our proposed algorithm yields perfect detection on both change points and anomaly points.
Revised: Figure 3 shows that on this dataset, our proposed algorithm yields perfect detection on both change points and anomaly points.

Original: In Figure 3, the gray part indicates the 90% predictive interval for forecasting.
Revised: In Figure 3, the gray part indicates the 90% predictive interval for forecasting.

Original: [Figure]
Revised: [Figure]

Original: We run our generative model 100 times to produce 100 different time series, and implement multiply methods on each of them, and aggregate the results together for comparison.
Revised: We run our generative model 100 times to produce 100 different time series, and implement multiply methods on each of them, and aggregate the results together for comparison.

Original: We include the following methodologies.
Revised: We include the following methodologies.

Original: For time series forecasting, we compare our method against Bayesian
Revised: For time series forecasting, we compare our method against Bayesian

Original: Structural Time Series (BSTS) (Scott & Varian, 2014; Brodersen et al., 2015)), Seasonal Decomposition of Time Series by Loess (STL) (Cleveland et al., 1990)), Seasonal ARIMA (Box et al., 2015), Holt–Winters (Holt, 2004), Exponential Smoothing State Space Model (ETS) (Hyndman et al., 2008)), and the Prophet R package by Taylor & Letham (2017).
Revised: Structural Time Series (BSTS) (Scott & Varian, 2014; Brodersen et al., 2015)), Seasonal Decomposition of Time Series by Loess (STL) (Cleveland et al., 1990)), Seasonal ARIMA (Box et al., 2015), Holt–Winters (Holt, 2004), Exponential Smoothing State Space Model (ETS) (Hyndman et al., 2008)), and the Prophet R package by Taylor & Letham (2017).

Original: We evaluate the performances by mean absolute percentage error (MAPE), mean square error (MSE) and mean absolute error (MAE) on forecasting set.
Revised: We evaluate the performances by mean absolute percentage error (MAPE), mean square error (MSE) and mean absolute error (MAE) on forecasting set.

Original: The mathematical deﬁnition of these three criterion is given as follows.
Revised: The mathematical deﬁnition of these three criterion is given as follows.

Original: Let x 1 , x 2 , . . .
Revised: Let x 1 , x 2 , . . .

Original: , x n be the true value and ˆ x 1 , ˆ x 2 , . .
Revised: , x n be the true value and ˆ x 1 , ˆ x 2 , . .

Original: , ˆ x n be the estimation or predictive values.
Revised: , ˆ x n be the estimation or predictive values.

Original: Then we have
Revised: Then we have

Original: [Equation]
Revised: [Equation]

Original: The comparison of our proposed algorithm and the aforementioned algorithms are included below in Table 2.
Revised: The comparison of our proposed algorithm and the aforementioned algorithms are included below in Table 2.

Original: As we mentioned in Section 6, our algorithm is downward compatible with the cases ignoring the existence of change point or anomaly, by setting p c = 0 or p a = 0 .
Revised: As we mentioned in Section 6, our algorithm is downward compatible with the cases ignoring the existence of change point or anomaly, by setting p c = 0 or p a = 0 .

Original: We also run proposed algorithm on the synthetic data with p c
Revised: We also run proposed algorithm on the synthetic data with p c

Original: = 

Revised: =

Original: (no change point), or p a = 

Revised: (no change point), or p a =

Original: no anomaly point), or p c = p a = 

Revised: no anomaly point), or p c = p a =

Original: (no change and anomaly point), for the purpose of numeric comparison.
Revised: (no change and anomaly point), for the purpose of numeric comparison.

Original: [Table] From Table 2 it turns out that our proposed algorithm achieves the best performance compared to other existing methods.
Revised: From Table 2 it turns out that our proposed algorithm achieves the best performance compared to other existing methods.

Original: Our proposed algorithm also performs better compared with the cases ignoring change point or anomaly point.
Revised: Our proposed algorithm also performs better compared with the cases ignoring change point or anomaly point.

Original: This is a convincing evidence on the importance of incorporating both change point structure and anomaly point structure when modeling, for time series forecasting.
Revised: This is a convincing evidence on the importance of incorporating both change point structure and anomaly point structure when modeling, for time series forecasting.

Original: We also compare our proposed method with other existing change point detection methods and anomaly detection algorithm with respect to the performance of detections.
Revised: We also compare our proposed method with other existing change point detection methods and anomaly detection algorithm with respect to the performance of detections.

Original: We evaluate the performance by two criterions: True Positive Rate (TPR) and False Positive (FP).
Revised: We evaluate the performance by two criterions: True Positive Rate (TPR) and False Positive (FP).

Original: TPR measures the percentage of change points or anomalies to be correctly detected.
Revised: TPR measures the percentage of change points or anomalies to be correctly detected.

Original: FP count the number of points wrongly detected as change points or anomaly points.
Revised: FP count the number of points wrongly detected as change points or anomaly points.

Original: The mathematical deﬁnitions of TPR and FP are as follows.
Revised: The mathematical deﬁnitions of TPR and FP are as follows.

Original: Let ( z 1 , z 2 , . . .
Revised: Let ( z 1 , z 2 , . . .

Original: , z n ) be the true binary vector for change points or anomalies, and  (ˆ z 1 , ˆ z 2 , . . 
Revised: , z n ) be the true binary vector for change points or anomalies, and [Table] (ˆ z 1 , ˆ z 2 , . . .

Original: , ˆ z n ) are the estimated ones.
Revised: , ˆ z n ) are the estimated ones.

Original: Then
Revised: Then

Revised: TPR =

Revised: |{ i : z i = 1 , ˆ z i = 1 }| |{ i : z i = 1 }| ,

Revised: FP = |{ i : z i = 0 , ˆ z i = 1 }| .

Revised: From the deﬁnition

Original: [Equation]
Revised: [Table]

Original: From the deﬁnition, we can see high TPR and low FP means the algorithm has better performance in detection.
Revised: we can see high TPR and low FP means the algorithm has better performance in detection.

Original: The comparison on change point detection is shown in Table 3.
Revised: The comparison on change point detection is shown in Table 3.

Original: We compare our results against three popular change point detection methods: Bayesian Change Point (BCP) (Barry & Hartigan, 1993), Change-Point (CP) (Killick & Eckley, 2014) and Breakout (twitter, 2017).
Revised: We compare our results against three popular change point detection methods: Bayesian Change Point (BCP) (Barry & Hartigan, 1993), Change-Point (CP) (Killick & Eckley, 2014) and Breakout (twitter, 2017).

Original: From Table 3 our proposed method outperforms the most of the others by both TPR and FP.
Revised: From Table 3 our proposed method outperforms the most of the others by both TPR and FP.

Original: We have smaller TPR compared to CP, but we are better in FP.
Revised: We have smaller TPR compared to CP, but we are better in FP.

Original: In Table 4, we also compare the performance of our algorithm on anomaly detection with three existing common anomaly detection methods: the AnomalyDetection package by Twitter (2017),
Revised: [Table]

Original: [Table]
Revised: [Table]

Revised: In Table

Original: [Table]
Revised: [Table]

Revised: we also compare the performance of our algorithm on anomaly detection with three existing common anomaly detection methods: the AnomalyDetection package by Twitter (2017),

Original: RAD by Netﬂix (2017) and Tsoutlier by Chen & Liu (1993).
Revised: RAD by Netﬂix (2017) and Tsoutlier by Chen & Liu (1993).

Original: The comparison is listed in Table 4.
Revised: The comparison is listed in Table [Table]

Original: We can see our method also outperforms most of the others with respect to anomaly detection, by both TPR and FP.
Revised: We can see our method also outperforms most of the others with respect to anomaly detection, by both TPR and FP.

Original: RAD has slightly better TPR but its FP is much worse compared with ours.
Revised: RAD has slightly better TPR but its FP is much worse compared with ours.

Original:  R EAL D ATA A NALYSIS
Revised:  R EAL D ATA A NALYSIS

Original: In this section, we implement our proposed method on real-world datasets.
Revised: In this section [Table] we implement our proposed method on real-world datasets.

Original: We also compare its performance against other existing time series forecasting methodologies.
Revised: We also compare its performance against other existing time series forecasting methodologies.

Original: We consider two datasets, one is a public data called Well-log dataset, and the other is an unpublished internet trafﬁc dataset.
Revised: We consider two datasets, one is a public data called Well-log dataset, and the other is an unpublished internet trafﬁc dataset.

Original: The bottom panels of Figure 4 and Figure 5 give the result of our proposed algorithms.
Revised: The bottom panels of Figure 4 and Figure 5 give the result of our proposed algorithms.

Original: The blue line separates the training set and testing set.
Revised: The blue line separates the training set and testing set.

Original: We use red line to show our ﬁtting and forecasting result, vertical red dashed line to indicate change points and purpledots to indicate anomaly points.
Revised: We use red line to show our ﬁtting and forecasting result, vertical red dashed line to indicate change points and purple

Original: The gray part shows 90% predication interval.

Original: 8. W ELL - LOG D ATA
Revised: 8. W ELL - LOG D ATA

Original: This dataset (Fearnhead & Clifford, 2003; JK & WJ, 1996) was collected when drilling a well.
Revised: This dataset (Fearnhead & Clifford, 2003; JK & WJ, 1996) was collected when drilling a well.

Original: It measures the nuclear magnetic response, which provides geophysical information to analyze the structure of rock surrounding the well.
Revised: It measures the nuclear magnetic response, which provides geophysical information to analyze the structure of rock surrounding the well.

Original: This dataset is public and available online 1 .
Revised: This dataset is public and available online 2 .

Original: It haspoints in total.
Revised: It has points in total.

Original: We split it such that the ﬁrst 3000 points are used as training set and last 1000 points are used to evaluate the forecasting performance.
Revised: We split it such that the ﬁrst 3000 points are used as training set and last 1000 points are used to evaluate the forecasting performance.

Original: [Figure]
Revised: [Figure]

Original: From Figure 4, it is obvious that there exists no seasonality or slope structure in the dataset.
Revised: From Figure 4, it is obvious that there exists no seasonality or slope structure in the dataset.

Original: This motivates us not to include these two components in our model.
Revised: This motivates us not to include these two components in our model.

Original: We implement our proposed algorithm without seasonality and slope, and compare the forecasting performance with other methods in Table 5.
Revised: We implement our proposed algorithm without seasonality and slope, and compare the forecasting performance with other methods in Table 5.

Original: Our method outperforms BSTS, ARIMA, ETS and Prophet.
Revised: Our method outperforms BSTS, ARIMA, ETS and Prophet.

Original: However in Table 5 the performance can be slightly improved if we ignore the existence of anomaly points by letting p a = 0 .
Revised: However in Table 5 the performance can be slightly improved if we ignore the existence of anomaly points by letting p a = 0 .

Original: This may be caused by model mis-speciﬁcation as the data may not generated in a way not entirely captured by our model.
Revised: This may be caused by model mis-speciﬁcation as the data may not generated in a way not entirely captured by our model.

Original: Nevertheless, the performances of our method considering anomaly points or not, are comparable to each other.
Revised: Nevertheless, the performances of our method considering anomaly points or not, are comparable to each other.

Revised: [Table]

Original: In this dataset there is no ground-truth of change point and anomaly point on their locations or even existence.
Revised: In this dataset there is no ground-truth of change point and anomaly point on their locations or even existence.

Original: However, from bottom panel of Figure 4, there are some obvious changes in the sequence and they all successfully captured by our algorithm.[Table]
Revised: However, from bottom panel of Figure 4, there are some obvious changes in the sequence and they all successfully captured by our algorithm.

Original: 8. I NTERNET T RAFFIC D ATA
Revised: 8. I NTERNET T RAFFIC D ATA

Original: Our second real data is an Internet trafﬁc data acquired from a major Tech company (see Figure 5).
Revised: Our second real data is an Internet trafﬁc data acquired from a major Tech company (see Figure 5).

Original: It is a daily trafﬁc data, with seasonality S = 7 .
Revised: It is a daily trafﬁc data, with seasonality S = 7 .

Original: We use the ﬁrst 800 observations as training set and evaluate the performance of forecasting on the remaining 265 points.
Revised: We use the ﬁrst 800 observations as training set and evaluate the performance of forecasting on the remaining 265 points.

Original: The bottom panel of Figureshow the result from implementing our algorithm
Revised: The bottom panel of Figureshow the result from implementing our algorithm.

Original: [Table]

Original: [Figure]
Revised: [Figure]

Original: We also do the comparison of forecasting performance of our proposed algorithm together with other existing methods, shown in Table 6.
Revised: We also do the comparison of forecasting performance of our proposed algorithm together with other existing methods, shown in Table 6.

Original: We can also see that our algorithm outperforms all the other algorithms with respect to MAPE, MSE and MAE.
Revised: We can also see that our algorithm outperforms all the other algorithms with respect to MAPE, MSE and MAE.

Original: [Table]
Revised: [Table]

Original: From Figure 5 our proposed algorithm identiﬁes one change point (the 576th point, indicated by the vertical red dashed line), which can be conﬁrmed that this is exactly the only one change point existing in this time series caused by the change of counting methods, by some external information.
Revised: From Figure 5 our proposed algorithm identiﬁes one change point (the 576th point, indicated by the vertical red dashed line), which can be conﬁrmed that this is exactly the only one change point existing in this time series caused by the change of counting methods, by some external information.

Original: Thus, we give the perfect change point detection in this Internet trafﬁc data.
Revised: Thus, we give the perfect change point detection in this Internet trafﬁc data.

Original: For this Internet trafﬁc dataset, since we have ground-truth for change point, we can compare the performance of change point detection of different methodologies.
Revised: For this Internet trafﬁc dataset, since we have ground-truth for change point, we can compare the performance of change point detection of different methodologies.

Original: BCP returns posterior distribution, which peaks in the the 576th point with posterior probability value 0.5.
Revised: BCP returns posterior distribution, which peaks in the the 576th point with posterior probability value 0.5.

Original: And it also returns with many other points with posterior probability value around 0.1.
Revised: And it also returns with many other points with posterior probability value around 0.1.

Original: CP returns 4 change points, where the 576th point (the only true one) is one of them.
Revised: CP returns 4 change points, where the 576th point (the only true one) is one of them.

Original: Breakout returns 8 change points without including the 576th point.
Revised: Breakout returns 8 change points without including the 576th point.

Original: To sum up, our proposed method achieves the best change point detection in this real dataset.  
Revised: To sum up, our proposed method achieves the best change point detection in this real dataset.  R ELATED W ORK

Revised: Parametric models are widely considered in econometric literature for time series forecasting, e.g. Jalles (2009), Commandeur et al.

Revised: (2011), Gould et al.

Revised: (2008), Harvey & Peters (1990), Harvey et al.

Revised: The general procedure of decomposition method (using trend, seasonal and irregular components) for univariate structural time series modeling is discussed in Harvey & Peters (1990); a uniﬁed state space framework is proposed to handle any messy time series in Harvey et al.

Original: C ONCLUSION
Revised: (1998); and the explicit modeling of both additive and multiplicative seasonalities in Gould et al. Jalles (2009). Although Kalman ﬁlter and MCMC-based approaches are used to sample posterior to estimate hidden components, the changing points and anomalies are not considered and processed in the above works. For example, the irregular component considered in Jalles (2009) is simply the noises.

Revised: Commandeur et al.

Revised: (2011) discusses the statistical software for state-space modeling which is designed for generic time series analytic and modeling, which cannot directly be used when changing point and anomalies are in existence.

Revised: Our proposed approach shares similarity with the aforementioned papers as we have similar additive structure of components.

Revised: However we are able to incorporate the change points and anomalies, two common structure widely observed in real data, into our model by using Bernoulli indicators.

Revised: This is non-trivial, and cannot be handled by the aforementioned papers or their variants.

Revised: Non-parametric approaches are used for extraction of components from quasi-periodic time-series, e.g., the ensembles of weak detectors using non-parametric measurement are used in Artemov &

Revised: Burnaev (2016) to detect change-points and anomalies, and the online decomposition algorithm based on per-component is adopted for change-point detection in Alexey Artemov (2015).

Revised: Different from the above works, to handle the structural brakes and change-points, this paper presents the parametric approach for modeling anomalies and changing points by ﬁtting them in the state-space framework using approximate inference for forecasting path prediction.

Revised: Different Bayesian approaches are proposed for change-point detection, e.g., Adams & MacKay (2007) performs Bayesian change point detection from online inference by generating the distribution estimation of the next unseen datum in the sequence given only data already observed, and the

Revised: Bayesian Online CPD (BOCPD) algorithm proposed by Turner et al.

Revised: (2009) performs online prediction using hidden variable given the underlying predictive model (UPM) and the hazard function.

Revised: Compared to the aforementioned models, our work differs in Bayesian modeling which samples posterior to estimate hidden components given the independent Bernoulli priors of changing point and anomalies.

Revised:  C ONCLUSION

Original: We incorporate the change point structure and anomaly point structure into the classic space state time series model.
Revised: We incorporate the change point structure and anomaly point structure into the classic space state time series model.

Original: We provide a Bayesian scheme for inference and time series forecasting.
Revised: We provide a Bayesian scheme for inference and time series forecasting.

Original: We compare the performance of our methodology and state-of-the-art methods on both synthetic data and real datasets.
Revised: We compare the performance of our methodology and state-of-the-art methods on both synthetic data and real datasets.

Original: Our method performs the best with respect to forecasting, change point detection, and anomaly detection as well.
Revised: Our method performs the best with respect to forecasting, change point detection, and anomaly detection as well.