Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposes a new deep learning architecture for multi-horizon forecasting, which aims to improve temporal representation learning using a bespoke Irregularity Representation Block.
strength_and_weaknesses: Strengths
---

Time series forecasting is a crucial problem across many application domains – with many datasets exhibiting level-trend-seasonality relationships in general. The development of models which can capture these relationships better would be very impactful for said applications.


Weaknesses
---

However, I do have several key concerns with the paper:

1. The paper does adopt some seemingly non-standard terms, which can be confusing with similar terms in general time series forecasting/representation learning literature.

     a.	Irregularity – the paper frequently describes residuals as “the irregularity”, without defining the term concretely or citing its source. Easily confused with modelling irregularly sampled time series – e.g. ICML 2022’s Modelling Irregular Time Series with Continuous Recurrent Units.

     b.	Reflection – appears to be used synonymously with “model”, and could be confused with concepts in vector transformations (https://en.wikipedia.org/wiki/Reflection_(mathematics)).

2. I am not sure I fully understand the motivations behind the architecture

     a.	Even if we were to understand “irregularity” as “residual”, the underlying model itself appears to be feeding in the inputs as-is – without any residualisation with respect to level-trend-seasonal components as implied by the summary.

     b.	The analogy of the IRN as a non-linear Fourier transform appears to breakdown when actual non-time inputs are used (i.e. x_input) – i.e. feeding a exogenous input into a sine wave would not necessarily make it a periodic basis function unless it is monotonic.

     c.	The description of the Fourier transform is also slightly unclear in equation 3

          i. Assuming $t_{a\~b}$ is a single point in time -- as implied by “When the irregularity … is observed at moment $t_{a\~b}$” – $min(t_{a\~b})$ becomes unmeaningful as it would simply equal $t_{a~b}$. If it represents a sequence, then sin(…) becomes non-meaningful.

     d. The parallels when we move into input space in equation 6 also require further explanation:

         i. Does dim(x^{input}) have to match dim(x^{irregular})? What if we only have a single input?

         ii. “Through this transformation, x^{irregular} is converted from time dependent to data dependent.”, but x^{input} does not appear to be a time index of any sort? Otherwise does dim d=1?

     e. How is the IRB used to generate multi-step forecasts in the table?

     f. How is hyperparameter optimisation performed for the model and benchmarks – details of which are crucial for reproducibility.

3. Unclear problem formulation – while the paper seemingly implies that the problem being tackled is for improving multi-horizon forecasts for datasets with strong level/trend/seasonal relationships, the precise problem is never fully defined. This makes it difficult to determine the appropriateness of the proposed benchmarks – many of which come from long-term time series forecasting (e.g. LogTran/Reformer/Pyraformer/Autoformer). Even with the simpler models – e.g. LSTM/GRU/VAR-MLP etc – how were they applied for multi-horizon forecasts?

     i. Assuming the paper is tackling models for enhanced level/trend/seasonality modelling – many existing papers do exist in the area that should be compared against (e.g ESRNN, Deep factors for forecasting etc) alongside simple econometric ETS models.

     ii. Assuming the paper is tackling models which used frequency-domain-based representations to improve time series forecasting, it should be compared against models such at the FEDformer for consistency.

clarity,_quality,_novelty_and_reproducibility: I would love to see the clarity of the paper improved, so as to perform a fuller assessment of the models merits -- with details in the section above.
summary_of_the_review: While I think the results show initial promise -- clarity with regards to the model's terminology, motivations and the problem it is tackling would need to be improved before it can be accepted.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper addresses the problem of time series forecasting.
The authors propose a model that combines convolutions and
min, max and average poolings, aiming to mimick a first order
approximation of a Fourier transformation. In experiments
they show that their method outperforms existing baselines.

strength_and_weaknesses: s1. very simple method.
s2. promising experimental results.

w1. misses a closely related, even more simple work.
w2. how the proposed model mimicks the Fourier transform is not clear.
w3. the concept of irregularity and the derivation of the approximation 
  is unclear.

clarity,_quality,_novelty_and_reproducibility: The paper is mostly well written, but sufferes from
a. unexplained symbols in the formulas (e.g. "R_CAV" in eq. 7,
  you only explain "R"; what is "t_{a~b}" ?),
b. many typos, e.g.,
  p. 2 "models is"
  p. 2 "the increasing the number"

Smaller points:
- tab. 3 and 4 are hard to read and miss some information:
  - mixing MSE and MAE rows makes not so much sense, as you never want to compare
    them.
  - standard deviations would make it easier to assess if we see significant differences
    or noise.

The proposed method combines standard components like convolutions
and pooling layers, but the specific cell that is constructed from those
components is novel.

The reproducibility is medium (experiments explained clearly, but no code
provided).

summary_of_the_review: I see three issues with the paper:

w1. misses a closely related, even more simple work.

  The DLinear model by Zeng et al. 2022 proposes an even simpler model with
  comparable results to yours. What advantages does your model have?

w2. how the proposed model mimicks the Fourier transform is not clear.

  In eq. 9, you define a layer conv-relu-conv-tanh and claim that it approximates
  the sine basis function. But there is nothing in your layer that would be geared
  specifically to model a sine function, you could claim that it approximates any
  function. To me it looks like your derivation breaks here.

w3. the concept of irregularity and the derivation of the approximation 
  is unclear.

  You claim that there is a third component in time series analysis besides
  seasonality and trend, "irregularity". To me this looks wrong and seems to
  misunderstand what a trend usually models in time series: often it is just
  a linear trend, and then you are perfectly right, there is room for further
  basis functions. But generally, trends can be any non-periodic basis functions,
  e.g. exponentials or logarithms etc. Between periodic ("seasonality") and
  non-periodic ("trend") basis functions I do not see a third type of basis
  functions.

  The basis function you model in the end (eq. 5) actually **is** a simple
  seasonality, a single sine function.

  The derivation of eq. 5 is problematic: the higher order terms do not
  "correspond to the vertical shift", because they also are periodic. If you
  want to make an approximation argument here, that **locally** the
  higher order terms can be approximated by a constant, you need to be
  more specific, esp. you need to say explicitly that n-th regular waves
  are the waves with **slower** angular velocity (as your fig. 3a suggests, but
  you never say that). However, in Fourier analysis by default basis functions
  are order by **increasing** angular velocity for good reasons, because
  only if you go to arbitrary high velocities you have arbitrary good
  approximations.

  So both, your concept of irregularity and the derivation of your layer
  function look problematic to me.


references:
  Ailing Zeng, Muxi Chen, Lei Zhang, Qiang Xu:
  Are Transformers Effective for Time Series Forecasting? CoRR abs/2205.13504 (2022)
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Summary: While time-series components like trends or seasonality have been extensively studied and been modeled in existing literature,
 	modeling residual or irregularity has been an under explored research area due to its inherent complexity. 
	In this work, the authors propose a deep learning based model for time-series forecasting (TSF) that reflects this irregularity
 	by introducing a novel expression for residual and employing it in TSF.
strength_and_weaknesses: Strength:
1. Residual modeling of time-series data is quite under explored which gives the research direction of this literature great value. 
2. Authors performed extensive experiments and ablation studies on number of real-world dataset to prove the validity of their approach

Weakness:
1. The authors didn’t provide any link to their code for reproducibility purpose, so it won’t be easy for peers to validate the experimental results
2. The technical depth of the work is a tad limited
clarity,_quality,_novelty_and_reproducibility: See Summary of the review
summary_of_the_review: Short review:
1. The paper’s presentation is decent and comprehensible
2. This work focuses on an under explored research area and shows great result when applied in real-world dataset
3. The paper doesn’t show the best technical depth 
4. No link to the code available
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a deep learning model for time series forecasting that reflects the irregularity in time series. Irregularity is based on Fourier series and and the authors employ it to design the Irregularity Representation Block  that captures, preserves, and learns the irregularity representation of time series data. Experiments on a variety of realworld datasets show the efficiency of the method.


strength_and_weaknesses: Mathematical models are used to explain the methodology and extensive experiments on a variety of real word datasets are provided. 

The code of the work is not available making difficult to evaluate if the result can be reproduced easily

clarity,_quality,_novelty_and_reproducibility: The paper is well written overall 
summary_of_the_review: The paper is interesting overall with mathematical models helping to understand the main concepts. Several experiments are provided to show the efficiency of the method, but we do not have access to the code.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper presents a method to model the residuals in time-series forecasting. Although this is an important problem to study, the paper confuses several well-defined terminologies - For eg., residuals have been defined as irregularity and model has been defined as reflections [Reviewer Lsd6]. 

Authors have not addressed concerns raised by reviewer Lsd6 that raised concerns about the suitability of the architecture used and problem formulation.
justification_for_why_not_higher_score: While the subject of the paper is interesting, it could benefit from clear motivations, problem formulation and architectural justifications.
justification_for_why_not_lower_score: The subject of modeling residuals in time-series data is interesting
Decision: Reject