Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a deep learning model tailor-designed for periodic time series forecasting problem. The inner architecture is composed of an expansion module to tackle complicated dependencies between inherent periods and time series signals and a periodicity module to capture sophisticated period signals. 
main_review: In general, this paper is well-written and easy to follow. Periodicity is a property rarely explicitly handled by previous deep learning models. The proposed model achieves better performance compared to previous methods such as N-BEATS. Experiments on different settings and parameters are provided. 

However, I have the following questions:
(1). Why does the expansion module need periodic blocks? Considering estimated hidden state z_t has already included inherent periodicity (assumed by the paper), why not directly taking x_t minus z_t to the local block for the predictions?

(2). The authors discuss the connections between the model and N-BEATS. What is unclear is the main difference between N-BEATS (interpretable) and DEPTS, since both of them have applied the periodicity. More illustrations should be provided. 

(3). Other time series analysis baselines such as ARIMA should be included in the experiments, although they may not beat the proposed method. 
summary_of_the_review: Overall the paper appears to have good quality, with a new time series prediction model proposed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper focuses on an important property of time series, periodicity, and mainly studies the problem of periodic time series (PTS) forecasting. This work solves two main research challenges of PTS forecasting: (1) to learn the dependencies of observation data on periodicity; (2) to learn the sophisticated compositions of various periods. The authors propose a deep expansion framework on top of residue learning for dependency learning and a parameterized surrogate function for periodicity learning. Extensive experiments on both synthetic data and 5 real-world datasets demonstrate a significant improvement on time series forecasting tasks when considering periodicity especially.
main_review: This paper brings us more attention to the periodicity modeling of time series forecasting in deep learning scenarios. After carefully checking the experiments on synthetic data (Figure 3 and Figure 7), I have noticed that even adding simple operations and periods on the auto-regressive signal data can lead to the state-of-the-art model (N-BEATS) degrade. This is quite meaningful and signifies the importance of periodicity on pure DL-based forecasting models. I think such simple yet intuitive observation can bring up good insights.

The paper has proposed a new decoupled formulation of time series forecasting and formulate the forecasting problem in the model DEPTS by introducing a prior estimated hidden periodic state and a expansion framework based on residue learning. Actually, this formulation is very interesting because most existing works are only to sample series based on the time window for training, which naturally fail to consider given series as whole and thus fail to capture all the information of a series globally. The proposed formulation cleverly makes use of periodicity to relieve this problem by the hidden periodic state, as those low-frequency periods need such modeling in particular. DEPTS framework builds upon the residual learning and expand the learned local residuals and periodic residuals block by block. The periodic blocks and local blocks process the periodicity and the local momenta respectively through expansion. In brief, the whole architecture design is straightforward and it can also be interpretable by outputs of different blocks.

This paper seems to have solid, extensive and diverse experiments. The extensive experimental results have clearly demonstrated the paper intuition (synthetic experiments), the performance improvement (real-world experiments) and the interpretability. The real-world experiments are on 5 different datasets, in which they follow settings of previous work on 3 datasets and construct 2 new datasets. The proposed model achieves better performances across all the datasets. Especially, I find out that DEPTS has a 10% improvement for M4 competition compared with state-of-the-art N-BEATS, which could very challenging. The sufficient interpretability cases are showed in each dataset. Overall, the experiment part is convincing.

Also, I do have some concerns. My first concern is about periodic module. As the paper states, the expansion module and the periodicity module jointly work for the forecasting based on many stacked layers. Each layer includes a local block and a periodic block for the residual expansion. The periodicity module is started with a parameter initialization, which is firstly initialized on the observation data. So my question is if the data is with an obvious trend (e.g., upward trend), how does the periodicity work since it can only handle periods with accumulated cosine functions? My another concern is with the training of the periodic module. If the periodic module is randomly initialized, can this DEPTS model still perform well? Would it still can accurately capture the obvious periods? I am very curious about the forecasting results without initialization of the periodicity module. 
summary_of_the_review: The paper solves the periodic time series forecasting problem; the proposed DEPTS model is intuitive, effective and interpretable; the experiment is solid and extensive; the improvement is significant.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper addresses the problem of time series forecasting, esp.
with periodic dependencies. The authors propose a model that combines
a learnt one-dimensional sum of cosines as periodic signal with
residual feedforward neural network (N-BEATS). They propose to
learn the model in two stages: first to estimate the cosines
with a discrete cosine transform and greedily selecting the
K ones with largest amplitude, second to refine both the
periodic time encoding and the overall forecasting model
end-to-end. In experiments on synthetic, three real-life datasets
from the literature and two new real-life datasets with longer
training segments, that should allow to identify periodicities
more easily, they show that they outperform the underlying
N-BEATS model mostly consistently. 

main_review: The paper is very well written and easy to follow. The proposed
model is plausible and the experiments show a lift. The paper is
accompagnied by a detailed appendix with further experiments,
esp. ablation studies. Methodologically, the paper is a
small extension of N-BEATS, but due to modelling the periodicity
explicitly an interesting one.

However, two questions about the proposed model are not
answered in the paper currently:

1. Does learning the periodic time encoding g_phi and the forecasting
   model f_theta really have advantages? One could imagine a simple
   baseline, where the learnt periodic time encoding is fed as covariate
   channel into N-BEATS. Is this as good as the proposed model?
   -- Also an ablation study in which the time encoder g_phi is frozen
   after the first stage, could shed some light on this question.

2. It is not clear on what hyperparameter grid the hyperparameter
   optimization has been conducted. Esp. in table 4 and 6, for N-BEATS
   and the proposed model the same backbone architecture seems to
   be optimal, while the proposed model gets additional complexity
   through the time encoder. Have N-BEATS architectures with a similar
   number of parameters as the optimal architecture for the proposed
   model been inside the hyperparameter grid for N-BEATS?


Small questions:
- what is the role of the two-stage optimization problem in eq. 5? If
  I understand it correctly, this problem never is tackled, but the
  authors use the heuristic fitting outlined in the two bullet points
  below eq. 5. Why do we need eq. 5 then?

summary_of_the_review: A well written paper with a small, but interesting contribution. 
Some aspects of the evaluation still can be made more clear.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposed a novel DL framework for periodic time series forecasting. The contributions of the paper include modeling complicated periodic dependency and capturing compositions of diversified periods. The authors conduct extensive experiments on synthetic data and real-world datasets, showing the effectiveness and interpretability of the proposed methods
main_review: ### strong points
1. a customized deep learning (DL) architecture for periodic time series to explicitly take account of the periodic property
2. the proposed techniques are reasonable and sound novel.
3. extensive experiments on a synthetic dataset and several real-world datasets about the effectiveness and interpretability.

### weak points
1. The writing of the paper should be improved. 
2. Some technical details are missing and some motivations are not very clear.
3. lack of significant test
4. lack of complexity analysis
summary_of_the_review: In this paper, the authors proposed a customized deep learning (DL) architecture for periodic time series to explicitly take account of the periodic property. The proposed solution is technically sound and effective based on the extensive evaluation. The periodical module sounds novel and reasonable, and parameter initialization resorts to a two-stage optimization problem, which is also interesting.  However, I have the following comments that the authors should take into account to improve the paper.

1. the writing should be improved. For example, Section 4.2 is not easy to read and the left part of Figure 2 is not very easy to understand. The authors are suggested to overview the framework before delving into the detailed description. The figure should be made more clear to understand. 
2. a few technical parts are missing. For example, how to design local block function f^l and periodic block function f^p. 
3. some motivations are not very clear. For example, why instantiate the period function as a series of cosine functions and how such a formulation can lead to the modeling of periodic patterns. 
4. Though the results of the proposed method are good, but in some cases, the improvements are small. The authors are suggested to include significant testing and show some standard error in the result table.
5.  Though the proposed method performs well, it is unclear the efficiency of the proposed method compared to baselines. The authors are suggested to add detailed complexity analysis to the proposed methods and some related baselines.
review_title: null

Meta-review: The paper proposed a novel deep learning model specifically designed for periodic time series forecasting problems. The approach includes lay-by-layer expansion, residual learning, and periodic parametrization. The model outperforms state-of-the-art baselines on several time series forecasting benchmarks.  The reviewers appreciate the extensive experimental results, but also suggested improvement on writing and comparison regarding the parameter efficiency of the model.
Decision: Accept (Spotlight)