Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper introduces a Bayesian model for timeseries with anomaly and change points besides regular trend and seasonality. It develops algorithms for inference and forecasting. The performance is evaluated and compared against state-of-the-art methods on three data sets: 1) synthetic data obtained from the generative Bayesian model itself; 2) well-log data; 3) internet traffic data.

On the methodological side, this appears to be a solid and significant contribution, although I am not sure how well it is aligned with the scope of ICLR. The introduced model is elegant; the algorithms for inference are non-trivial.

From a practical perspective, one cannot expect this contribution to be ground-breaking, since there has been more than 40 years of work on time series forecasting, change point and anomaly detection. In some situations the methodology proposed here will work better than previous approaches (particularly in the situation where the data comes from the Bayesian model itself - in that case, there clearly is no better approach), in other cases (which the paper might have put less emphasis on), previous approaches will work better. To position this kind of work, I think it is important the authors discuss the limitations of their approach. Some guidelines on when or when not to use it would be valuable. Clearly, these days one cannot introduce methodology in this area and expect it to outperform existing methods under all circumstances (and hence practitioners to always choose it over any other existing method).

What is surprising is that relatively simple approaches like ETS or STL work pretty much equally well (in some cases even better in terms of MSE) than the proposed approach, while more recent approaches - like BSTS - dramatically fail. It would be good if the authors could comment on why this might be the case.

Summary:
+ Methodology appears to be a significant, solid contribution.
- Experiments are not conclusive as to when or when not to choose this approach over existing methods
- writing needs to be improved (large number of grammatical errors and typos, e.g. 'Mehtods')
review_title: Solid methodology; its practical performance requires further investigation

final_score: rating: 5: Marginally below acceptance threshold
review_content: 

Summary:

This paper develops a state space time series forecasting model in the Bayesian framework, jointly detects anomaly and change points. Integrated with an iterative MCMC method, the authors develop an efficient algorithm and use both synthetic and real data set to demonstrate that their algorithms outperform many other state-of-art algorithms. 

Major comments:
In the beginning of section 3, the authors assume that all the terms that characterize the change-points and anomaly points are normally distributed with mean zero and different variance. However, in classic formulation for change-point or anomaly detection, usually there is also a mean shift other than the variance change. For example, we might assume $r_t \sim N(\theta, \sigma_r^2)$ for some $\theta>0$ to demonstrate the positive mean shift. I believe that this kind of mean shift is more efficient to model the structure of change-point. 

My main concern is with the novelty. The work does not seem to be very novel.

Minor comments:

1. In the end of the page 2, the last panel is the residual, not the spikes. 

2. In page 12, the caption of figure 5 should be (left) and (right), not (top) and (bottom).
review_title: Review for “BAYESIAN TIME SERIES FORECASTING WITH CHANGE POINT AND ANOMALY DETECTION”

final_score: rating: 4: Ok but not good enough - rejection
review_content: Minor comments:
- page 3. “The observation equation and transition equations together (i.e., Equation (1,2,3)) together define “ - one “together” should be removed
- page 4. “From Figure 2, the joint distribution (i.e., the likelihood function ” - there should be additional bracket
- page 7. “We can further integral out αn “ -> integrate out

Major comments:
The paper is well-written. The paper considers structural time-series model with seasonal component and stochastic trend, which allow for change-points and structural breaks.

Such type of parametric models are widely considered in econometric literature, see e.g.
[1] Jalles, João Tovar, Structural Time Series Models and the Kalman Filter: A Concise Review (June 19, 2009). FEUNL Working Paper No. 541. Available at SSRN: https://ssrn.com/abstract=1496864 or http://dx.doi.org/10.2139/ssrn.1496864 
[2] Jacques J. F. Commandeur, Siem Jan Koopman, Marius Ooms. Statistical Software for State Space Methods // May 2011, Volume 41, Issue 1.
[3] Scott, Steven L. and Varian, Hal R., Predicting the Present with Bayesian Structural Time Series (June 28, 2013). Available at SSRN: https://ssrn.com/abstract=2304426 or http://dx.doi.org/10.2139/ssrn.2304426 
[4] Phillip G. Gould, Anne B. Koehler, J. Keith Ord, Ralph D. Snyder, Rob J. Hyndman, Farshid Vahid-Araghi, Forecasting time series with multiple seasonal patterns, In European Journal of Operational Research, Volume 191, Issue 1, 2008, Pages 207-222, ISSN 0377-2217, https://doi.org/10.1016/j.ejor.2007.08.024.
[5] A.C. Harvey, S. Peters. Estimation Procedures for structural time series models // Journal of Forecasting, Vol. 9, 89-108, 1990
[6] A. Harvey, S.J. Koopman, J. Penzer. Messy Time Series: A Unified approach // Advances in Econometrics, Vol. 13, pp. 103-143.

They also use Kalman filter and MCMC-based approaches to sample posterior to estimate hidden components.

There are also non-parametric approaches to extraction of components from quasi-periodic time-series, see e.g.
[7] Artemov A., Burnaev E. Detecting Performance Degradation of Software-Intensive Systems in the Presence of Trends and Long-Range Dependence // 16th International Conference on Data Mining Workshops (ICDMW), IEEE Conference Publications, pp. 29 - 36, 2016. DOI: 10.1109/ICDMW.2016.0013
[8] Alexey Artemov, Evgeny Burnaev and Andrey Lokot. Nonparametric Decomposition of Quasi-periodic Time Series for Change-point Detection // Proc. SPIE 9875, Eighth International Conference on Machine Vision, 987520 (December 8, 2015); 5 P. doi:10.1117/12.2228370;http://dx.doi.org/10.1117/12.2228370

In some of these papers models of structural brakes and change-points are also considered, see e.g. 
- page 118 in [6]
- papers [7, 8]

There were also Bayesian approaches for change-point detection, which are similar to the model of change-point, proposed in the considered paper, e.g.
[9] Ryan Prescott Adams, David J.C. MacKay. Bayesian Online Changepoint Detection // https://arxiv.org/abs/0710.3742
[10] Ryan Turner, Yunus Saatçi, and Carl Edward Rasmussen. Adaptive sequential Bayesian change point detection. In Zaïd Harchaoui, editor, NIPS Workshop on Temporal Segmentation, Whistler, BC, Canada, December 2009.

Thus,
- the paper does not provide comparison with relevant econometric literature on parametric structural time-series models,
- the paper does not provide comparison with relevant advanced change-point detection methods e.g. [7,8,9,10]. The comparison is provided only with very simple methods,
- the proposed model itself looks very similar to what can be found across econometric literature,
- the datasets, used for comparison, are very scarce. There are datasets for anomaly detection in time-series data, which should be used for extensive comparison, e.g. Numenta Anomaly Detection Benchmark.

Therefore, also the paper is well-written, 
- it lacks novelty,
- its topic does not perfectly fit topics of interest for ICLR,
So, I do not recommend this paper to be published.
review_title: Well-written paper, but lack of novelty

Meta-review: Thank you for submitting you paper to ICLR.  The consensus from the reviewers is that this is not quite ready for publication. There is also concern about whether ICLR, with its focus on representational learning, is the right venue for this work.

One of the reviewers initially submitted an incorrect review, but this mistake has now been rectified. Apologies that this was not done sooner in order to allow you to address their concerns.
Decision: Reject