Original: N-BEATS: N EURAL BASIS EXPANSION ANALYSIS FOR INTERPRETABLE TIME SERIES FORECASTING
Revised: N-BEATS: N EURAL BASIS EXPANSION ANALYSIS FOR INTERPRETABLE TIME SERIES FORECASTING

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: We focus on solving the univariate times series point forecasting problem using deep learning.
Revised: We focus on solving the univariate times series point forecasting problem using deep learning.

Original: We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers.
Revised: We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers.

Original: The architecture has a number of desirable properties, being interpretable, applicable without modiﬁcation to a wide array of target domains, and fast to train.
Revised: The architecture has a number of desirable properties, being interpretable, applicable without modiﬁcation to a wide array of target domains, and fast to train.

Original: We test the proposed architecture on several well-known datasets, including M3, M4 and
Revised: We test the proposed architecture on several well-known datasets, including M3, M4 and

Original: TOURISM competition datasets containing time series from diverse domains.
Revised: TOURISM competition datasets containing time series from diverse domains.

Original: We demonstrate state-of-the-art performance for two conﬁgurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year’s winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models.
Revised: We demonstrate state-of-the-art performance for two conﬁgurations of N-BEATS for all the datasets, improving forecast accuracy by 11% over a statistical benchmark and by 3% over last year’s winner of the M4 competition, a domain-adjusted hand-crafted hybrid between neural network and statistical time series models.

Original: The ﬁrst conﬁguration of our model does not employ any time-series-speciﬁc components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufﬁcient to solve a wide range of forecasting problems.
Revised: The ﬁrst conﬁguration of our model does not employ any time-series-speciﬁc components and its performance on heterogeneous datasets strongly suggests that, contrarily to received wisdom, deep learning primitives such as residual blocks are by themselves sufﬁcient to solve a wide range of forecasting problems.

Original: Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.
Revised: Finally, we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Time series (TS) forecasting is an important business problem and a fruitful application area for machine learning (ML).
Revised: Time series (TS) forecasting is an important business problem and a fruitful application area for machine learning (ML).

Original: It underlies most aspects of modern business, including such critical areas as inventory control and customer management, as well as business planning going from production and distribution to ﬁnance and marketing.
Revised: It underlies most aspects of modern business, including such critical areas as inventory control and customer management, as well as business planning going from production and distribution to ﬁnance and marketing.

Original: As such, it has a considerable ﬁnancial impact, often ranging in the millions of dollars for every point of forecasting accuracy gained (Jain, 2017; Kahn, 2003).
Revised: As such, it has a considerable ﬁnancial impact, often ranging in the millions of dollars for every point of forecasting accuracy gained (Jain, 2017; Kahn, 2003).

Original: And yet, unlike areas such as computer vision or natural language processing where deep learning (DL) techniques are now well entrenched, there still exists evidence that ML and DL struggle to outperform classical statistical TS forecasting approaches (Makridakis et al., 2018a;b).
Revised: And yet, unlike areas such as computer vision or natural language processing where deep learning (DL) techniques are now well entrenched, there still exists evidence that ML and DL struggle to outperform classical statistical TS forecasting approaches (Makridakis et al., 2018a;b).

Original: For instance, the rankings of the six “pure” ML methods submitted to M4 competition were 23, 37, 38, 48, 54, and 57 out of a total of 60 entries, and most of the best-ranking methods were ensembles of classical statistical techniques (Makridakis et al., 2018b).
Revised: For instance, the rankings of the six “pure” ML methods submitted to M4 competition were 23, 37, 38, 48, 54, and 57 out of a total of 60 entries, and most of the best-ranking methods were ensembles of classical statistical techniques (Makridakis et al., 2018b).

Original: On the other hand, the M4 competition winner, an approach by S. Smyl, was based on a hybrid between neural residual/attention dilated LSTM stack with a classical Holt-Winters statistical model (Holt, 1957; 2004; Winters, 1960) with learnable parameters.
Revised: On the other hand, the M4 competition winner, an approach by S. Smyl, was based on a hybrid between neural residual/attention dilated LSTM stack with a classical Holt-Winters statistical model (Holt, 1957; 2004; Winters, 1960) with learnable parameters.

Original: Since Smyl’s approach heavily depends on this Holt-Winters component, Makridakis et al.
Revised: Since Smyl’s approach heavily depends on this Holt-Winters component, Makridakis et al.

Original: (2018b) further argue that “hybrid approaches and combinations of method are the way forward for improving the forecasting accuracy and making forecasting more valuable”.
Revised: (2018b) further argue that “hybrid approaches and combinations of method are the way forward for improving the forecasting accuracy and making forecasting more valuable”.

Original: In this work we aspire to challenge this conclusion by exploring the potential of pure DL architectures in the context of the TS forecasting.
Revised: In this work we aspire to challenge this conclusion by exploring the potential of pure DL architectures in the context of the TS forecasting.

Original: Moreover, in the context of interpretable DL architecture design, we are interested in answering the following question: can we inject a suitable inductive bias in the model to make its internal operations more interpretable, in the sense of extracting some explainable driving factors combining to produce a given forecast?
Revised: Moreover, in the context of interpretable DL architecture design, we are interested in answering the following question: can we inject a suitable inductive bias in the model to make its internal operations more interpretable, in the sense of extracting some explainable driving factors combining to produce a given forecast?

Original: 1. S UMMARY OF C ONTRIBUTIONS
Revised: 1. S UMMARY OF C ONTRIBUTIONS

Original: Deep Neural Architecture:
Revised: Deep Neural Architecture:

Original: To the best of our knowledge, this is the ﬁrst work to empirically demonstrate that pure DL using no time-series speciﬁc components outperforms well-established statistical approaches on M3, M4 and TOURISM datasets (on M4, by 11% over statistical benchmark, by 7% over the best statistical entry, and by 3% over the M4 competition winner).
Revised: To the best of our knowledge, this is the ﬁrst work to empirically demonstrate that pure DL using no time-series speciﬁc components outperforms well-established statistical approaches on M3, M4 and TOURISM datasets (on M4, by 11% over statistical benchmark, by 7% over the best statistical entry, and by 3% over the M4 competition winner).

Original: In our view, this provides a long-missing proof of concept for the use of pure ML in TS forecasting and strengthens motivation to continue advancing the research in this area.
Revised: In our view, this provides a long-missing proof of concept for the use of pure ML in TS forecasting and strengthens motivation to continue advancing the research in this area.

Original: Interpretable DL for Time Series:
Revised: Interpretable DL for Time Series:

Original: In addition to accuracy beneﬁts, we also show that it is feasible to design an architecture with interpretable outputs that can be used by practitioners in very much the same way as traditional decomposition techniques such as the “seasonality-trend-level” approach (Cleveland et al., 1990).
Revised: In addition to accuracy beneﬁts, we also show that it is feasible to design an architecture with interpretable outputs that can be used by practitioners in very much the same way as traditional decomposition techniques such as the “seasonality-trend-level” approach (Cleveland et al., 1990).

Original:  P ROBLEM S TATEMENT
Revised:  P ROBLEM S TATEMENT

Original: We consider the univariate point forecasting problem in discrete time.
Revised: We consider the univariate point forecasting problem in discrete time.

Original: Given a length- H forecast horizon a length- T observed series history [ y 1 ,..., y T ] ∈ R T , the task is to predict the vector of future values y ∈ R H =
Revised: Given a length- H forecast horizon a length- T observed series history [ y 1 ,..., y T ] ∈ R T , the task is to predict the vector of future values y ∈ R H =

Original: [ y T + 1 , y T + 2 ,..., y T + H ] .
Revised: [ y T + 1 , y T + 2 ,..., y T + H ] .

Original: For simplicity, we will later consider a lookback window of length t ≤ T ending with the last observed value y T to serve as model input, and denoted x ∈ R t =
Revised: For simplicity, we will later consider a lookback window of length t ≤ T ending with the last observed value y T to serve as model input, and denoted x ∈ R t =

Original: [ y T − t + 1 ,..., y T ] .
Revised: [ y T − t + 1 ,..., y T ] .

Original: We denote (cid:98) y
Revised: We denote (cid:98) y

Original: the forecast of y .
Revised: the forecast of y .

Original: The following metrics are commonly used to evaluate forecasting performance (Hyndman & Koehler, 2006; Makridakis & Hibon, 2000;
Revised: The following metrics are commonly used to evaluate forecasting performance (Hyndman & Koehler, 2006; Makridakis & Hibon, 2000;

Original: Makridakis et al., 2018b; Athanasopoulos et al., 2011):
Revised: Makridakis et al., 2018b; Athanasopoulos et al., 2011):

Original: [Equation]
Revised: [Equation]

Revised: y T

Revised: + i − (cid:98)

Revised: y T

Revised: + i | | y T + i | ,

Revised: MASE =

Revised: 1 H

Revised: H

Revised: ∑ i =| y T + i − (cid:98) y T

Revised: + i | 1 T

Revised: + H − m ∑ T

Revised: +

Revised: H j

Revised: = m + 1 | y j − y j − m | ,

Revised: OWA = 1(cid:20)

Original: {+s MAPE s MAPE Naïve
+}
Revised: {+s MAPE s MAPE Naïve
+}

Original:  Here m is the periodicity of the data ( e.g. , 12 for monthly series).
Revised: + MASE MASE Naïve (cid:21) . Here m is the periodicity of the data ( e.g. , 12 for monthly series).

Original: MAPE (Mean Absolute Percentage
Revised: MAPE (Mean Absolute Percentage

Original: Error), s MAPE (symmetric MAPE ) and MASE (Mean Absolute Scaled Error) are standard scale-free metrics in the practice of forecasting (Hyndman & Koehler, 2006; Makridakis & Hibon, 2000): whereas s MAPE scales the error by the average between the forecast and ground truth, the MASE scales by the average error of the naïve predictor that simply copies the observation measured
Revised: Error), s MAPE (symmetric MAPE ) and MASE (Mean Absolute Scaled Error) are standard scale-free metrics in the practice of forecasting (Hyndman & Koehler, 2006; Makridakis & Hibon, 2000): whereas s MAPE scales the error by the average between the forecast and ground truth, the MASE scales by the average error of the naïve predictor that simply copies the observation measured

Original: m periods in the past, thereby accounting for seasonality.
Revised: m periods in the past, thereby accounting for seasonality.

Original: OWA (overall weighted average) is a M4speciﬁc metric used to rank competition entries (M4 Team, 2018b), where s MAPE and MASE metrics are normalized such that a seasonally-adjusted naïve forecast obtains OWA = 1 . 0.
Revised: OWA (overall weighted average) is a M4speciﬁc metric used to rank competition entries (M4 Team, 2018b), where s MAPE and MASE metrics are normalized such that a seasonally-adjusted naïve forecast obtains OWA = 1 . 0.

Original:  N-BEATS
Revised:  N-BEATS

Original: Our architecture design methodology relies on a few key principles.
Revised: Our architecture design methodology relies on a few key principles.

Original: First, the base architecture should be simple and generic, yet expressive (deep).
Revised: First, the base architecture should be simple and generic, yet expressive (deep).

Original: Second, the architecture should not rely on timeseries-speciﬁc feature engineering or input scaling.
Revised: Second, the architecture should not rely on timeseries-speciﬁc feature engineering or input scaling.

Original: These prerequisites let us explore the potential of pure DL architecture in TS forecasting.
Revised: These prerequisites let us explore the potential of pure DL architecture in TS forecasting.

Original: Finally, as a prerequisite to explore interpretability, the architecture should be extendable towards making its outputs human interpretable.
Revised: Finally, as a prerequisite to explore interpretability, the architecture should be extendable towards making its outputs human interpretable.

Original: We now discuss how those principles converge to the proposed architecture.
Revised: We now discuss how those principles converge to the proposed architecture.

Original: 3. B ASIC B LOCK
Revised: 3. B ASIC B LOCK

Original: The proposed basic building block has a fork architecture and is depicted in Fig.
Revised: The proposed basic building block has a fork architecture and is depicted in Fig.

Original: 1 (left).
Revised: 1 (left).

Revised: We focus on describing the operation of (cid:96) -th block in this section in detail (note that the block index (cid:96) is dropped in Fig.

Original: It accepts an input x  and outputs two vectors, (cid:98) x  and(cid:98) y .
Revised: 1 for brevity). The (cid:96) -th block accepts its respective input x (cid:96) and outputs two vectors, (cid:98) x (cid:96) and

Original: Input x  is  a history lookback window of certain length ending with the last measured observation, (cid:98) y is the block’s forward forecast of length H , and (cid:98) x is the block’s best estimate of x , also known as the ‘backcast’, given the constraints on the functional space that the block can use to approximate signals.
Revised: (cid:98) y (cid:96) . For the very ﬁrst block in the model, its respective x (cid:96) is the overall model input — a history lookback window of certain length ending with the last measured observation.

Original: We set the length of input window to a multiple of the forecast horizon H , and typical lengths of x in our setup range from 2 H to 7 H .
Revised: We set the length of input window to a multiple of the forecast horizon H , and typical lengths of x in our setup range from 2 H to 7 H .

Revised: For the rest of the blocks, their inputs x (cid:96) are residual outputs of the previous blocks.

Revised: Each block has two outputs: (cid:98) y (cid:96) , the block’s forward forecast of length H ; and (cid:98) x

Revised: (cid:96) , the block’s best estimate of x (cid:96) , also known as the ‘backcast’, given the constraints on the functional space that the block can use to approximate signals.

Original: Internally, the basic building block consists of two parts.
Revised: Internally, the basic building block consists of two parts.

Original: First, the waveform generator g θ : T N (cid:55)→ Y N is a map from N points in  the time domain T N ⊂ R N to N points in the forecast (or backcast)
Revised: The ﬁrst part is a fully connected network that produces the forward θ f (cid:96) and the backward θ b (cid:96) predictors of expansion coefﬁcients (again, note that the block index (cid:96) is dropped for θ b (cid:96) , θ f (cid:96) , g b (cid:96) , g f (cid:96) in Fig. 1 for brevity). The second part consists of the backward g b (cid:96) and the forward g f (cid:96) basis layers that accept the respective forward θ f (cid:96) and backward θ b (cid:96) expansion coefﬁcients, project them internally on the set of basis functions and produce the backcast (cid:98) x (cid:96) and the forecast outputs (cid:98) y (cid:96) deﬁned in the previous paragraph. The operation of the ﬁrst part of the (cid:96) -th block is described by the following equations:

Original: [Figure]
Revised: [Equation]

Original: value domain, Y N ⊂ R N .
Revised: Here L INEAR layer is simply a linear projection layer, i.e. θ f (cid:96) = W f (cid:96) h (cid:96), 4 .

Original: The number of points in forecast  and backcast is different, in general.
Revised: The FC layer is a standard fully connected layer with R E L U non-linearity (Nair & Hinton, 2010), such that for FC (cid:96), 1 we have, for example: h (cid:96), 1 = R E L U ( W (cid:96), 1 x (cid:96) + b (cid:96), 1 ) . One task of this part of the architecture is to predict the forward expansion coefﬁcients θ f (cid:96) with the ultimate goal of optimizing the accuracy of the partial forecast (cid:98) y (cid:96) by properly mixing the basis vectors supplied by g f (cid:96) . Additionally, this sub-network predicts backward expansion coefﬁcients θ b (cid:96) used by g b (cid:96) to produce an estimate of x (cid:96) with the ultimate goal of helping the downstream blocks by removing components of their input that are not helpful for forecasting. The second part of the network maps expansion coefﬁcients θ f (cid:96) and θ b (cid:96) to outputs via basis layers, (cid:98) y (cid:96) =

Original: The waveform generator is parameterized with θ ∈ Θ ⊂ R M . 
Revised: g f (cid:96) ( θ f (cid:96) ) and (cid:98) x (cid:96) = g b (cid:96) ( θ b (cid:96) ) . Its operation is described by the following equations:

Original:  The function of g θ is to providea sufﬁciently rich set of time-varying waveforms, selectable by varying θ . As shown below, g θ can either be chosen to be learnable or can be set to speciﬁc funtional forms to reﬂect certain problem-speciﬁc inductive biases in order to appropriately constrain the structure of outputs.
Revised: [Equation] Here v fi and v bi are forecast and backcast basis vectors, θ f (cid:96), i is the i -th element of θ f (cid:96) . The function of g b (cid:96) and g f (cid:96) is to provide sufﬁciently rich sets { v fi } dim ( θ f

Original: Concrete examples of g θ are discussed in Section 3.3. Second, the forward and the backward predictors of parameters of the waveform generator are maps ϕ f φ : R dim ( x ) (cid:55)→ Θ and ϕ b φ : R dim ( x ) (cid:55)→ Θ .
Revised: (cid:96) ) i =and { v bi } dim ( [Equation]

Original: The task of ϕ f φ is to predict the forward expansion parameters θ f with the ultimate goal of optimizing the accuracy of the partial forecast (cid:98) y by properly mixing the waveforms supplied by g θ  .
Revised: b (cid:96) [Equation] i =such that their respective outputs can be represented adequately via varying expansion coefﬁcients θ f (cid:96) and θ b (cid:96) .

Original: Conversely, the task of ϕ b φ is to produce an estimate of x with the ultimate goal of helping the downstream blocks by removing components of their input that are not helpful for forecasting. We use ϕ φ that has a form of the multi-layer fully connected (FC) neural network with ReLU non-linearity (Nair & Hinton, 2010).
Revised: As shown below, g b (cid:96) and g f (cid:96) can either be chosen to be learnable or can be set to speciﬁc functional forms to reﬂect certain problem-speciﬁc inductive biases in order to appropriately constrain the structure of outputs.

Original: The number of hidden layers in the network is 4 (see Appendix E for detailed hyperparameter settings).
Revised: Concrete examples of g b (cid:96) and g f (cid:96) are discussed in Section 3.3.

Original: 3. D OUBLY R ESIDUAL S TACKING
Revised: 3. D OUBLY R ESIDUAL S TACKING

Original: The classical residual network architecture adds the input of the stack of layers to its output before passing the result to the next stack (He et al., 2016).
Revised: The classical residual network architecture adds the input of the stack of layers to its output before passing the result to the next stack (He et al., 2016).

Original: The DenseNet architecture proposed by Huang et al.
Revised: The DenseNet architecture proposed by Huang et al.

Original: (2017) extends this principle by introducing extra connections from the output of each stack to the input of every other stack that follows it.
Revised: (2017) extends this principle by introducing extra connections from the output of each stack to the input of every other stack that follows it.

Original: These approaches provide clear advantages in improving the trainability of deep architectures.
Revised: These approaches provide clear advantages in improving the trainability of deep architectures.

Original: Their disadvantage in the context of this work is that they result in network structures that are difﬁcult to interpret.
Revised: Their disadvantage in the context of this work is that they result in network structures that are difﬁcult to interpret.

Original: We propose a novel hierarchical doubly residual topology depicted in Fig.
Revised: We propose a novel hierarchical doubly residual topology depicted in Fig.

Original: 1 (middle and right).
Revised: 1 (middle and right).

Original: The proposed architecture has two residual branches, one running over backcast predictionbranch of each layer and the other one is running over the forecast branch of each layer. 
Revised: The proposed architecture has two residual branches, one running over backcast prediction of each layer and the other one is running over the forecast branch of each layer. Its operation is described by the following equations:

Revised: [Equation]

Revised: As previously mentioned, in the special case of the very ﬁrst block, its input is the model level input x , x 1 ≡ x .

Original: The backcast residual branch  can be thought of as running a sequential analysis of the input signal.
Revised: For all other blocks, the backcast residual branch x (cid:96) can be thought of as running a sequential analysis of the input signal.

Original: Each block removes the portion of the signal  that it can explain well, making the forecast job of the downstream blocks easier.
Revised: Previous block removes the portion of the signal (cid:98) x (cid:96) − 1 that it can approximate well, making the forecast job of the downstream blocks easier.

Original: This structure also facilitates more ﬂuid gradient backpropagation.
Revised: This structure also facilitates more ﬂuid gradient backpropagation.

Original: More importantly, each layer outputs a partial  forecast  that is ﬁrst aggregated at the stack level and then at the overall network level, providing a hierarchical decomposition.
Revised: More importantly, each block outputs a partial [Figure] forecast (cid:98) y (cid:96) that is ﬁrst aggregated at the stack level and then at the overall network level, providing a hierarchical decomposition.

Revised: The ﬁnal forecast (cid:98) y is the sum of all partial forecasts.

Original: In a generic model context, when stacks are allowed to have arbitrary g θ for each layer, this makes the network more transparent to gradient ﬂows.
Revised: In a generic model context, when stacks are allowed to have arbitrary g b (cid:96) and g f (cid:96) for each layer, this makes the network more transparent to gradient ﬂows.

Original: In a special situation of deliberate structure enforced in g θ shared over a stack, explained next, this has the critical importance of enabling interpretability via the aggregation of meaningful partial forecasts.
Revised: In a special situation of deliberate structure enforced in g b (cid:96) and g f (cid:96) shared over a stack, explained next, this has the critical importance of enabling interpretability via the aggregation of meaningful partial forecasts.

Original: 3. I NTERPRETABILITY
Revised: 3. I NTERPRETABILITY

Original: We propose two conﬁgurations of the architecture, based on the selection of g θ .
Revised: We propose two conﬁgurations of the architecture, based on the selection of g b (cid:96) and g f (cid:96) .

Original: One of them is generic DL, the other one is augmented with certain inductive biases to be interpretable.
Revised: One of them is generic DL, the other one is augmented with certain inductive biases to be interpretable.

Original: The generic architecture does not rely on TS-speciﬁc knowledge.
Revised: The generic architecture does not rely on TS-speciﬁc knowledge.

Original: We set g θ to be a linear projection of the previous layer output.
Revised: We set g b (cid:96) and g f (cid:96) to be a linear projection of the previous layer output.

Original: In this case the partial forecast at the output of block j in stack i is:
Revised: In this case the outputs of block (cid:96) are described as:

Original: [Equation]
Revised: [Equation]

Original: The interpretation of this model is that the FC layers in the basic building block depicted in Fig.learn the predictive decomposition of the partial forecast (cid:98) y i , j in the basis W i , j learned by the network.
Revised: The interpretation of this model is that the FC layers in the basic building block depicted in Fig. 1 learn the predictive decomposition of the partial forecast (cid:98) y (cid:96) in the basis V f (cid:96) learned by the network.

Original: Matrix W i , j has dimensionality H × dim ( θ i , j ) .
Revised: Matrix V f (cid:96) has dimensionality H × dim ( θ f (cid:96) ) .

Original: Therefore, the ﬁrst dimension of W i , j has the interpretation of discrete time index in the forecast domain.
Revised: Therefore, the ﬁrst dimension of V f (cid:96) has the interpretation of discrete time index in the forecast domain.

Original: The second dimension of the matrix has the interpretation of the indices of the basis functions, with θ i , j being the expansion coefﬁcients for this basis.
Revised: The second dimension of the matrix has the interpretation of the indices of the basis functions, with θ f (cid:96) being the expansion coefﬁcients for this basis.

Original: Thus the columns of W i , j can be thought of as waveforms in the time domain.
Revised: Thus the columns of V f (cid:96) can be thought of as waveforms in the time domain.

Original: Because no additional constraints are imposed on the form of W i , j , the waveforms learned by the deep model do not have inherent structure (and none is apparent in our experiments).This leads to (cid:98) y i , j as well as stack outputs (cid:98) y i
Revised: Because no additional constraints are imposed on the form of V f (cid:96) , the waveforms learned by the deep model do not have inherent structure (and none is apparent in our experiments).

Original: = ∑ j (cid:98) y i , j not being interpretable.
Revised: This leads to (cid:98) y (cid:96) not being interpretable.

Original: The interpretable architecture can be constructed by reusing the overall architectural approach in Fig.
Revised: The interpretable architecture can be constructed by reusing the overall architectural approach in Fig.

Original: 1 and by adding structure to g θ for each stack.
Revised: 1 and by adding structure to basis layers at stack level.

Original: Forecasting practitioners often use the decomposition of time series into trend and seasonality, such as those performed by the STL (Cleveland et al., 1990) and X 13- ARIMA (U.S. Census Bureau, 2013).
Revised: Forecasting practitioners often use the decomposition of time series into trend and seasonality, such as those performed by the STL (Cleveland et al., 1990) and X 13- ARIMA (U.S. Census Bureau, 2013).

Original: We propose to design the trend and seasonality decomposition into the model to make the stack outputs more easily interpretable.
Revised: We propose to design the trend and seasonality decomposition into the model to make the stack outputs more easily interpretable.

Revised: Note that for the generic model the notion of stack was not necessary and the stack level indexing was omitted for clarity.

Revised: Now we will consider both stack level and block level indexing.

Revised: For example, (cid:98) y s ,(cid:96) will denote the partial forecast of block (cid:96) within stack s .

Original: Trend model.
Revised: Trend model.

Original: A typical characteristic of trend is that most of the time it is a monotonic function, or at least a slowly varying function.
Revised: A typical characteristic of trend is that most of the time it is a monotonic function, or at least a slowly varying function.

Original: In order to mimic this behaviour we propose to constrain g θ to be a polynomial of small degree p , a function slowly varying across forecast window:
Revised: In order to mimic this behaviour we propose to constrain g bs ,(cid:96) and g fs ,(cid:96) to be a polynomial of small degree p , a function slowly varying across forecast window:

Original: [Equation]
Revised: [Equation]

Original: Here time vector t = [ 0 , 1 , 2 ,..., H − 2 , H − 1 ] T / H is deﬁned on a discrete grid running from 0 to ( H − 1 ) / H , forecasting H steps ahead.
Revised: Here time vector t = [ 0 , 1 , 2 ,..., H − 2 , H − 1 ] T / H is deﬁned on a discrete grid running from 0 to ( H − 1 ) / H , forecasting H steps ahead.

Original: The trend forecast  will then have the form:
Revised: Alternatively, the trend forecast in matrix form will then be:

Original: [Equation]
Revised: [Equation]

Original: where θ i , j are polynomial coefﬁcients predicted by a FC network of layer j of stack i and T = [ 1 , t ,..., t p ] is the matrix of powers of t .
Revised: where θ fs ,(cid:96) are polynomial coefﬁcients predicted by a FC network of layer (cid:96) of stack s described by equations (1); and T = [ 1 , t ,..., t p ] is the matrix of powers of t .

Original: If p is low, e.g. 2 or 3, it forces (cid:98) y tri , j to mimic trend.
Revised: If p is low, e.g. 2 or 3, it forces (cid:98) y trs ,(cid:96) to mimic trend.

Original: Seasonality model.
Revised: Seasonality model.

Original: Typical characteristic of seasonality is that it is a regular, cyclical, recurring ﬂuctuation.
Revised: Typical characteristic of seasonality is that it is a regular, cyclical, recurring ﬂuctuation.

Original: Therefore, to model seasonality, we propose to constrain g θ to belong to the class of periodic functions g θ ( t) =
Revised: Therefore, to model seasonality, we propose to constrain g bs ,(cid:96) and g fs ,(cid:96) to belong to the class of periodic functions, i.e. y t =

Original: g θ ( t − s ) , where s is a seasonality period.
Revised: y t − ∆ , where ∆ is a seasonality period.

Original: A natural choice for the basis to model periodic function is the Fourier series:
Revised: A natural choice for the basis to model periodic function is the Fourier series:

Original: [Equation]
Revised: [Equation]

Original: The seasonality forecast will then have the form:
Revised: The seasonality forecast will then have the matrix form as follows:

Original: [Equation]
Revised: [Equation]

Original: where θ i , j are Fourier coefﬁcients predicted by a FC network of layer j of stack i and S =
Revised: where θ fs ,(cid:96) are Fourier coefﬁcients predicted by a FC network of layer (cid:96) of stack s described by equations (1); and S =

Original: [ 1 , cos ( 2 π t ) ,... cos ( 2 π (cid:98) H /
Revised: [ 1 , cos ( 2 π t ) ,... cos ( 2 π (cid:98) H /

Original: 2 − 1 (cid:99) t )) , sin ( 2 π t ) ,..., sin ( 2 π (cid:98) H / 2 − 1 (cid:99) t ))] is the matrix of sinusoidal waveforms.
Revised: 2 − 1 (cid:99) t )) , sin ( 2 π t ) ,..., sin ( 2 π (cid:98) H / 2 − 1 (cid:99) t ))] is the matrix of sinusoidal waveforms.

Original: The forecast (cid:98) y si , j is then a periodic function mimicking typical seasonal patterns.
Revised: The forecast (cid:98) y seass ,(cid:96) is then a periodic function mimicking typical seasonal patterns.

Original: The overall interpretable architecture consists of two stacks only: the trendmodeling stack is followed by the seasonalitymodeling stack.
Revised: The overall interpretable architecture consists of two stacks: the trend stack is followed by the seasonality stack.

Original: The doubly residual stacking combined with the forecast/backcast principle result in (i) the trend component being removed from the input window x before it is fed into the seasonalityanalysis stack and (ii) the partial forecasts of trend and seasonality are available as separate interpretable outputs.
Revised: The doubly residual stacking combined with the forecast/backcast principle result in (i) the trend component being removed from the input window x before it is fed into the seasonality stack and (ii) the partial forecasts of trend and seasonality are available as separate interpretable outputs.

Original: Structurally, each of the stacks consists of several blocks connected with residual connections as depicted in Fig.
Revised: Structurally, each of the stacks consists of several blocks connected with residual connections as depicted in Fig.

Original: 1 and each of them shares its respective, non-learnable g θ .
Revised: 1 and each of them shares its respective, non-learnable g bs ,(cid:96) and g f s ,(cid:96) .

Original: The number of blocks is 3 for both trend and seasonality.
Revised: The number of blocks is 3 for both trend and seasonality.

Original: We found that on top of sharing g θ , sharing all the weights across blocks in a stack resulted in betterperformance on validation set.
Revised: We found that on top of sharing g bs ,(cid:96) and g fs ,(cid:96) , sharing all the weights across blocks in a stack resulted in better validation performance.

Original:  E NSEMBLING
Revised: 3. E NSEMBLING

Original: Ensembling is used by all the top entries in the M4-competition.
Revised: Ensembling is used by all the top entries in the M4-competition.

Original: We rely on ensembling as well to be comparable.
Revised: We rely on ensembling as well to be comparable.

Original: We found that ensembling is a much more powerful regularization technique than the popular alternatives, e.g. dropout or L2-norm penalty.
Revised: We found that ensembling is a much more powerful regularization technique than the popular alternatives, e.g. dropout or L2-norm penalty.

Original: The addition of those methods improved individual models, but was hurting the performance of the ensemble.
Revised: The addition of those methods improved individual models, but was hurting the performance of the ensemble.

Original: The core property of an ensemble is diversity.
Revised: The core property of an ensemble is diversity.

Original: We build an ensemble using several sources of diversity.
Revised: We build an ensemble using several sources of diversity.

Original: First, the ensemble models are ﬁt on three different metrics: s MAPE , MASE and MAPE , a version of s MAPE that has only the ground truth value in the denominator.
Revised: First, the ensemble models are ﬁt on three different metrics: s MAPE , MASE and MAPE , a version of s MAPE that has only the ground truth value in the denominator.

Original: Second, for every horizon H , individual models are trained on input windows of different length: 2 H , 3 H ,..., 7 H , for a total of six window lengths.
Revised: Second, for every horizon H , individual models are trained on input windows of different length: 2 H , 3 H ,..., 7 H , for a total of six window lengths.

Original: Thus the overall ensemble exhibits a multi-scale aspect.
Revised: Thus the overall ensemble exhibits a multi-scale aspect.

Original: Finally, we perform a bagging procedure (Breiman, 1996) by including models trained with different random initializations.
Revised: Finally, we perform a bagging procedure (Breiman, 1996) by including models trained with different random initializations.

Original: We use 180 total models to report results on the test set (please refer to Appendix C for the ablation of ensemble size).
Revised: We use 180 total models to report results on the test set (please refer to Appendix C for the ablation of ensemble size).

Original: We use the median as ensemble aggregation function. 
Revised: We use the median as ensemble aggregation function. [Table]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: The approaches to TS forecasting can be split in a few distinct categories.
Revised: The approaches to TS forecasting can be split in a few distinct categories.

Original: The statistical modeling approaches based on exponential smoothing and its different ﬂavors are well established and are often considered a default choice in the industry (Holt, 1957; 2004; Winters, 1960).
Revised: The statistical modeling approaches based on exponential smoothing and its different ﬂavors are well established and are often considered a default choice in the industry (Holt, 1957; 2004; Winters, 1960).

Original: More advanced variations of exponential smoothing include the winner of M3 competition, the Theta method (Assimakopoulos & Nikolopoulos, 2000) that decomposes the forecast into several theta-lines and statistically combines them.
Revised: More advanced variations of exponential smoothing include the winner of M3 competition, the Theta method (Assimakopoulos & Nikolopoulos, 2000) that decomposes the forecast into several theta-lines and statistically combines them.

Original: The pinnacle of the statistical approach encapsulates ARIMA, autoARIMA and in general, the uniﬁed state-space modeling approach, that can be used to explain and analyze all of the approaches mentioned above (see Hyndman & Khandakar (2008) for an overview).
Revised: The pinnacle of the statistical approach encapsulates ARIMA, autoARIMA and in general, the uniﬁed state-space modeling approach, that can be used to explain and analyze all of the approaches mentioned above (see Hyndman & Khandakar (2008) for an overview).

Original: More recently, ML/TS combination approaches started inﬁltrating the domain with great success, showing promising results by using the outputs of statistical engines as features.
Revised: More recently [Table] ML/TS combination approaches started inﬁltrating the domain with great success, showing promising results by using the outputs of statistical engines as features.

Original: In fact, 2 out of topentries in the M4 competition are approaches of this type, including the second entry (Montero-Manso et al., 2019).
Revised: In fact, 2 out of topentries in the M4 competition are approaches of this type, including the second entry (Montero-Manso et al., 2019).

Original: The second entry computes the outputs of several statistical methods on the M4 dataset and combines them using gradient boosted tree (Chen & Guestrin, 2016).
Revised: The second entry computes the outputs of several statistical methods on the M4 dataset and combines them using gradient boosted tree (Chen & Guestrin, 2016).

Original: Somewhat independently, the work in the modern deep learning TS forecasting developed based on variations of recurrent neural networks (Flunkert et al., 2017; Rangapuram et al., 2018; Toubeau et al., 2019; Zia & Razzaq, 2018) being largely dominated by the electricity load forecasting in the multi-variate setup.
Revised: Somewhat independently, the work in the modern deep learning TS forecasting developed based on variations of recurrent neural networks (Flunkert et al., 2017; Rangapuram et al., 2018; Toubeau et al., 2019; Zia & Razzaq, 2018) being largely dominated by the electricity load forecasting in the multi-variate setup.

Original: A few earlier works explored the combinations of recurrent neural networks with dilation, residual connections and attention (Chang et al., 2017; Kim et al., 2017; Qin et al., 2017).
Revised: A few earlier works explored the combinations of recurrent neural networks with dilation, residual connections and attention (Chang et al., 2017; Kim et al., 2017; Qin et al., 2017).

Original: These served as a basis for the winner of the M4 competition, an approach developed by S. Smyl.
Revised: These served as a basis for the winner of the M4 competition, an approach developed by S. Smyl.

Original: The winning entry combines a
Revised: The winning entry combines a

Original: Holt-Winters style seasonality model with its parameters ﬁtted to a given TS via gradient descent and a unique combination of dilation/residual/attention approaches for each forecast horizon.
Revised: Holt-Winters style seasonality model with its parameters ﬁtted to a given TS via gradient descent and a unique combination of dilation/residual/attention approaches for each forecast horizon.

Original: The resulting model is a hybrid model that architecturally heavily relies on a time-series engine.
Revised: The resulting model is a hybrid model that architecturally heavily relies on a time-series engine.

Original: It is hand crafted to each speciﬁc horizon of M4, making this approach hard to generalize to other datasets.
Revised: It is hand crafted to each speciﬁc horizon of M4, making this approach hard to generalize to other datasets.

Original:  E XPERIMENTAL R ESULTS
Revised:  E XPERIMENTAL R ESULTS

Original: Our key empirical results based on aggregate performance metrics over several datasets—M4 (M
Revised: Our key empirical results based on aggregate performance metrics over several datasets—M4 (M

Original: Team, 2018b; Makridakis et al., 2018b), M3 (Makridakis & Hibon, 2000; Makridakis et al., 2018a) and TOURISM (Athanasopoulos et al., 2011)—appear in Table 1.
Revised: Team, 2018b; Makridakis et al., 2018b), M3 (Makridakis & Hibon, 2000; Makridakis et al., 2018a) and TOURISM (Athanasopoulos et al., 2011)—appear in Table 1.

Original: More detailed descriptions of the datasets are provided in Section 5.1 and Appendix B. For each dataset, we compare our results with best 5 entries for this dataset reported in the literature, according to the customary metrics speciﬁc to each dataset (M4: OWA and s MAPE , M3: s MAPE , TOURISM : MAPE ).
Revised: More detailed descriptions of the datasets are provided in Section 5.1 and Appendix B. For each dataset, we compare our results with best 5 entries for this dataset reported in the literature, according to the customary metrics speciﬁc to each dataset (M4: OWA and s MAPE , M3: s MAPE , TOURISM : MAPE ).

Original: More granular dataset-speciﬁc results with data splits over forecast horizons and types of time series appear in respective appendices (M4: Appendix D.1; M3: Appendix D.2; TOURISM : Appendix D.3).[Table]
Revised: More granular dataset-speciﬁc results with data splits over forecast horizons and types of time series appear in respective appendices (M4: Appendix D.1; M3: Appendix D.2; TOURISM : Appendix D.3).

Original: In Table [Table] we study the performance of two N-BEATS conﬁgurations: generic (N-BEATS-G) and interpretable (N-BEATS-I), as well as N-BEATS-I+G (ensemble of all models from N-BEATS-G and N-BEATS-I).
Revised: In Table 1, we study the performance of two N-BEATS conﬁgurations: generic (N-BEATS-G) and interpretable (N-BEATS-I), as well as N-BEATS-I+G (ensemble of all models from N-BEATS-G and N-BEATS-I).

Original: On M4 dataset [Table] we compare against 5 representatives from the M4 competition (Makridakis et al., 2018b): each best in their respective model class.
Revised: On M4 dataset , we compare against 5 representatives from the M4 competition (Makridakis et al., 2018b): each best in their respective model class.

Original: Pure ML is the submission by B. Trotta, the best entry among the 6 pure ML models.
Revised: Pure ML is the submission by B. Trotta, the best entry among the 6 pure ML models.

Original: Statistical is the best pure statistical model by N.Z. Legaki and K. Koutsouri.
Revised: Statistical is the best pure statistical model by N.Z. Legaki and K. Koutsouri.

Original: ML/TS combination is the model by P. Montero-Manso,
Revised: ML/TS combination is the model by P. Montero-Manso,

Original: T. Talagala, R.J. Hyndman and G. Athanasopoulos, second best entry, gradient boosted tree over a few statistical time series models.
Revised: T. Talagala, R.J. Hyndman and G. Athanasopoulos, second best entry, gradient boosted tree over a few statistical time series models.

Original: ProLogistica is the third entry in M4 based on the weighted ensemble of statistical methods.
Revised: ProLogistica is the third entry in M4 based on the weighted ensemble of statistical methods.

Original: Finally, DL/TS hybrid is the winner of M4 competition designed by S.
Revised: Finally, DL/TS hybrid is the winner of M4 competition designed by S.

Original: Smyl.
Revised: Smyl.

Original: On the M3 dataset , we compare against the Theta method (Assimakopoulos & Nikolopoulos, 2000), the winner of M3; DOTA , a dynamically optimized Theta model (Fiorucci et al., 2016);
Revised: On the M3 dataset , we compare against the Theta method (Assimakopoulos & Nikolopoulos, 2000), the winner of M3; DOTA , a dynamically optimized Theta model (Fiorucci et al., 2016);

Original: EXP , the most resent statistical approach and the previous state-of-the-art on M3 (Spiliotis et al., 2019); as well as ForecastPro , an off-the-shelf forecasting software that is based on model selection between exponential smoothing, ARIMA and moving average (Athanasopoulos et al., 2011;
Revised: EXP , the most resent statistical approach and the previous state-of-the-art on M3 (Spiliotis et al., 2019); as well as ForecastPro , an off-the-shelf forecasting software that is based on model selection between exponential smoothing, ARIMA and moving average (Athanasopoulos et al., 2011;

Original: Assimakopoulos & Nikolopoulos, 2000).
Revised: Assimakopoulos & Nikolopoulos, 2000).

Original: On the TOURISM dataset , we compare against 3 statistical benchmarks (Athanasopoulos et al., 2011): ETS , exponential smoothing with cross-validated additive/multiplicative model; Theta method; ForePro , same as ForecastPro in M3; as well as topentries from the TOURISM Kaggle competition (Athanasopoulos & Hyndman, 2011): Stratometrics , an unknown technique; LeeCBaker (Baker & Howard, 2011), a weighted combination of Naïve, linear trend model, and exponentially weighted least squares regression trend.
Revised: On the TOURISM dataset , we compare against 3 statistical benchmarks (Athanasopoulos et al., 2011): ETS , exponential smoothing with cross-validated additive/multiplicative model; Theta method; ForePro , same as ForecastPro in M3; as well as topentries from the TOURISM Kaggle competition (Athanasopoulos & Hyndman, 2011): Stratometrics , an unknown technique; LeeCBaker (Baker & Howard, 2011), a weighted combination of Naïve, linear trend model, and exponentially weighted least squares regression trend.

Original: According to Table 1, N-BEATS demonstrates state-of-the-art performance on three challenging non-overlapping datasets containing time series from very different domains, sampling frequencies and seasonalities.
Revised: According to Table 1, N-BEATS demonstrates state-of-the-art performance on three challenging non-overlapping datasets containing time series from very different domains, sampling frequencies and seasonalities.

Original: As an example, on M4 dataset, the OWA gap between N-BEATS and the M winner (0 . 821 − 0 .
Revised: As an example, on M4 dataset, the OWA gap between N-BEATS and the M winner (0 . 821 − 0 .

Original: 795 = 0 . 026) is greater than the gap between the M4 winner and the second entry (0 . 838 − 0 .
Revised: 795 = 0 . 026) is greater than the gap between the M4 winner and the second entry (0 . 838 − 0 .

Original: = 0 . 017).
Revised: = 0 . 017).

Original: Generic N-BEATS model uses as little prior knowledge as possible, with no feature engineering, no scaling and no internal architectural components that may be considered
Revised: Generic N-BEATS model uses as little prior knowledge as possible, with no feature engineering, no scaling and no internal architectural components that may be considered

Original: TS-speciﬁc.
Revised: TS-speciﬁc.

Original: Thus the result in Table 1 leads us to the conclusion that DL does not need support from the statistical approaches or hand-crafted feature engineering and domain knowledge to perform extremely well on a wide array of TS forecasting tasks.
Revised: Thus the result in Table 1 leads us to the conclusion that DL does not need support from the statistical approaches or hand-crafted feature engineering and domain knowledge to perform extremely well on a wide array of TS forecasting tasks.

Original: On top of that, the proposed general architecture performs very well on three different datasets outperforming a wide variety of models, both generic and manually crafted to respective dataset, including the winner of M4, a model architecturally adjusted by hand to each forecast-horizon subset of the M4 data.
Revised: On top of that, the proposed general architecture performs very well on three different datasets outperforming a wide variety of models, both generic and manually crafted to respective dataset, including the winner of M4, a model architecturally adjusted by hand to each forecast-horizon subset of the M4 data.

Original: 5. D ATASETS
Revised: 5. D ATASETS

Original: M4 (M4 Team, 2018b; Makridakis et al., 2018b) is the latest in an inﬂuential series of forecasting competitions organized by Spyros Makridakis since 1982 (Makridakis et al., 1982).
Revised: M4 (M4 Team, 2018b; Makridakis et al., 2018b) is the latest in an inﬂuential series of forecasting competitions organized by Spyros Makridakis since 1982 (Makridakis et al., 1982).

Original: The 100k-series dataset is large and diverse, consisting of data frequently encountered in business, ﬁnancial and economic forecasting, and sampling frequencies ranging from hourly to yearly.
Revised: The 100k-series dataset is large and diverse, consisting of data frequently encountered in business, ﬁnancial and economic forecasting, and sampling frequencies ranging from hourly to yearly.

Original: A table with summary statistics is presented in Appendix B.1, showing wide variability in TS characteristics.
Revised: A table with summary statistics is presented in Appendix B.1, showing wide variability in TS characteristics.

Original: M3 (Makridakis & Hibon, 2000) is similar in its composition to M4, but has a smaller overall scale (3003 time series total vs. 100k in M4).
Revised: M3 (Makridakis & Hibon, 2000) is similar in its composition to M4, but has a smaller overall scale (3003 time series total vs. 100k in M4).

Original: A table with summary statistics is presented in Appendix B.2.
Revised: A table with summary statistics is presented in Appendix B.2.

Original: Over the past 20 years, this dataset has supported signiﬁcant efforts in the design of more optimal statistical models, e.g. Theta and its variants (Assimakopoulos & Nikolopoulos, 2000; Fiorucci et al., 2016; Spiliotis et al., 2019).
Revised: Over the past 20 years, this dataset has supported signiﬁcant efforts in the design of more optimal statistical models, e.g. Theta and its variants (Assimakopoulos & Nikolopoulos, 2000; Fiorucci et al., 2016; Spiliotis et al., 2019).

Original: Furthermore, a recent publication (Makridakis et al., 2018a) based on a subset of M3 presented evidence that ML models are inferior to the classical statistical models.
Revised: Furthermore, a recent publication (Makridakis et al., 2018a) based on a subset of M3 presented evidence that ML models are inferior to the classical statistical models.

Original: TOURISM (Athanasopoulos et al., 2011) dataset was released as part of the respective Kaggle competition conducted by Athanasopoulos & Hyndman (2011).
Revised: TOURISM (Athanasopoulos et al., 2011) dataset was released as part of the respective Kaggle competition conducted by Athanasopoulos & Hyndman (2011).

Original: The data include monthly, quarterly and yearly series supplied by both governmental tourism organizations (e.g. Tourism Australia, the
Revised: The data include monthly, quarterly and yearly series supplied by both governmental tourism organizations (e.g. Tourism Australia, the

Original: Hong Kong Tourism Board and Tourism New Zealand) as well as various academics, who had used them in previous studies.
Revised: Hong Kong Tourism Board and Tourism New Zealand) as well as various academics, who had used them in previous studies.

Original: A table with summary statistics is presented in Appendix B.3.
Revised: A table with summary statistics is presented in Appendix B.3.

Original: 5. T RAINING METHODOLOGY
Revised: 5. T RAINING METHODOLOGY

Original: We split each dataset into train, validation and test subsets.
Revised: We split each dataset into train, validation and test subsets.

Original: The test subset is the standard test set previously deﬁned for each dataset (M4 Team, 2018a; Makridakis & Hibon, 2000; Athanasopoulos et al., 2011).
Revised: The test subset is the standard test set previously deﬁned for each dataset (M4 Team, 2018a; Makridakis & Hibon, 2000; Athanasopoulos et al., 2011).

Original: The validation and train subsets for each dataset are obtained by splitting their full train sets at the boundary of the last horizon of each time series.
Revised: The validation and train subsets for each dataset are obtained by splitting their full train sets at the boundary of the last horizon of each time series.

Original: We use the train and validation subsets to tune hyperparameters.
Revised: We use the train and validation subsets to tune hyperparameters.

Original: Once the hyperparameters are determined, we train the model on the full train set and report results on the test set.
Revised: Once the hyperparameters are determined, we train the model on the full train set and report results on the test set.

Revised: Please refer to Appendix E for detailed hyperparameter settings at the block level.

Original: N-BEATS is implemented and trained in Tensorﬂow (Abadi et al., 2015).
Revised: N-BEATS is implemented and trained in Tensorﬂow (Abadi et al., 2015).

Original: We share parameters of the network across horizons, therefore we train one model per horizon for each dataset.
Revised: We share parameters of the network across horizons, therefore we train one model per horizon for each dataset.

Original: If every time series is interpreted as a separate task, this can be linked back to the multitask learning and furthermore to meta-learning (see discussion in Appendix A), in which a neural network is regularized by learning on multiple tasks to improve generalization.
Revised: If every time series is interpreted as a separate task, this can be linked back to the multitask learning and furthermore to meta-learning (see discussion in Appendix A), in which a neural network is regularized by learning on multiple tasks to improve generalization.

Original: We would like to stress that models for different horizons and datasets reuse the same architecture.
Revised: We would like to stress that models for different horizons and datasets reuse the same architecture.

Original: Architectural hyperparameters (width, number of layers, number of stacks, etc.) are ﬁxed to the same values across horizons and across datasets (see Appendix E).
Revised: Architectural hyperparameters (width, number of layers, number of stacks, etc.) are ﬁxed to the same values across horizons and across datasets (see Appendix E).

Original: The fact that we can reuse architecture and even hyperparameters across horizons indicates that the proposed architecture design generalizes well across time series of different nature.
Revised: The fact that we can reuse architecture and even hyperparameters across horizons indicates that the proposed architecture design generalizes well across time series of different nature.

Original: The same architecture is successfully trained on the M Monthly subset with 48k time series and the M

Revised: The same architecture is successfully trained on the M4 Monthly subset with 48k time series and the M

Original: Others subset with 174 time series.
Revised: Others subset with 174 time series.

Original: This is a much stronger result than e.g. the result of S. Smyl (Makridakis et al., 2018b) who had to use very different architectures hand crafted for different horizons.
Revised: This is a much stronger result than e.g. the result of S. Smyl (Makridakis et al., 2018b) who had to use very different architectures hand crafted for different horizons.

Original: To update network parameters for one horizon, we sample train batches of ﬁxed size 1024.
Revised: To update network parameters for one horizon, we sample train batches of ﬁxed size 1024.

Original: We pick 1024 TS ids from this horizon, uniformly at random with replacement.
Revised: We pick 1024 TS ids from this horizon, uniformly at random with replacement.

Original: For each selected TS id we pick a random forecast point from the historical range of length L H immediately preceding the last point in the train part of the TS.
Revised: For each selected TS id we pick a random forecast point from the historical range of length L H immediately preceding the last point in the train part of the TS.

Original: L H is a cross-validated hyperparameter.
Revised: L H is a cross-validated hyperparameter.

Original: We observed that for subsets with large number of time series it tends to be smaller and for subsets with smaller number of time series it tends to be larger.
Revised: We observed that for subsets with large number of time series it tends to be smaller and for subsets with smaller number of time series it tends to be larger.

Original: For example, in massive Yearly, Monthly, Quarterly subsets of M4 L H is equal to 1 . 5; and in moderate to small Weekly, Daily, Hourly subsets of M4 L H is equal to 10.
Revised: For example, in massive Yearly, Monthly, Quarterly subsets of M4 L H is equal to 1 . 5; and in moderate to small Weekly, Daily, Hourly subsets of M4 L H is equal to 10.

Original: Given a sampled forecast point, we set one horizon worth of points following it to be the target forecast window y
Revised: Given a sampled forecast point, we set one horizon worth of points following it to be the target forecast window y

Original: and we set the history of points of one of lengths 2 H , 3 H ,..., 7 H preceding it to be the input x to the network.
Revised: and we set the history of points of one of lengths 2 H , 3 H ,..., 7 H preceding it to be the input x to the network.

Original: We use the Adam optimizer with default settings and initial learning rate 0.001.
Revised: We use the Adam optimizer with default settings and initial learning rate 0.001.

Original: The neural network training is run with early stopping and the number of batches is determined on the validation set.
Revised: The neural network training is run with early stopping and the number of batches is determined on the validation set.

Original: The GPU based training of one ensemble member for entire M4 dataset takes between 30 min and 2 hours depending on neural network settings and hardware.
Revised: The GPU based training of one ensemble member for entire M4 dataset takes between 30 min and 2 hours depending on neural network settings and hardware.

Original: 5. I NTERPRETABILITY RESULTS
Revised: 5. I NTERPRETABILITY RESULTS

Original: Fig.
Revised: Fig.

Original: 2 studies the outputs of the proposed model in the generic and the interpretable conﬁgurations.
Revised: 2 studies the outputs of the proposed model in the generic and the interpretable conﬁgurations.

Original: As discussed in Section 3.3, to make the generic architecture presented in Fig.
Revised: As discussed in Section 3.3, to make the generic architecture presented in Fig.

Original: 1 interpretable, we constrain g θ in the ﬁrst stack to have the form of polynomial (1) while the second one has the form of Fourier basis (2).
Revised: 1 interpretable, we constrain g θ in the ﬁrst stack to have the form of polynomial (2) while the second one has the form of Fourier basis (3).

Original: Furthermore, we use the outputs of the generic conﬁguration of N-BEATS as control group (the generic model of 30 residual blocks depicted in Fig.
Revised: Furthermore, we use the outputs of the generic conﬁguration of N-BEATS as control group (the generic model of 30 residual blocks depicted in Fig.

Original: 1 is divided into two stacks) and we plot both generic (sufﬁx “-G”) and interpretable (sufﬁx “-I”) stack outputs side by side in
Revised: 1 is divided into two stacks) and we plot both generic (sufﬁx “-G”) and interpretable (sufﬁx “-I”) stack outputs side by side in

Original: Fig.
Revised: Fig.

Original: The outputs of generic model are arbitrary and non-interpretable: either trend or seasonality or both of them are present at the output of both stacks.
Revised: The outputs of generic model are arbitrary and non-interpretable: either trend or seasonality or both of them are present at the output of both stacks.

Original: The magnitude of the output (peak-to-peak) is generally smaller at the output of the second stack.
Revised: The magnitude of the output (peak-to-peak) is generally smaller at the output of the second stack.

Original: The outputs of the interpretable model exhibit[Figure] distinct properties: the trend output is monotonic and slowly moving, the seasonality output is regular, cyclical and has recurring ﬂuctuations.
Revised: The outputs of the interpretable model exhibit distinct properties: the trend output is monotonic and slowly moving, the seasonality output is regular, cyclical and has recurring ﬂuctuations.

Original: The peak-to-peak magnitude of the seasonality output is signiﬁcantly larger than that of the trend, if signiﬁcant seasonality is present in the time series.
Revised: The peak-to-peak magnitude of the seasonality output is signiﬁcantly larger than that of the trend, if signiﬁcant seasonality is present in the time series.

Original: Similarly, the peak-to-peak magnitude of trend output tends to be small when no obvious trend is present in the ground truth signal.
Revised: Similarly, the peak-to-peak magnitude of trend output tends to be small when no obvious trend is present in the ground truth signal.

Original: Thus the proposed interpretable architecture decomposes its forecast into two distinct components.
Revised: Thus the proposed interpretable architecture decomposes its forecast into two distinct components.

Original: Our conclusion is that the outputs of the DL model can be made interpretable by encoding a sensible inductive bias in the architecture.
Revised: Our conclusion is that the outputs of the DL model can be made interpretable by encoding a sensible inductive bias in the architecture.

Original: Table 1 conﬁrms that this does not result in performance drop.
Revised: Table 1 conﬁrms that this does not result in performance drop.

Original:  C ONCLUSIONS
Revised:  C ONCLUSIONS

Original: We proposed and empirically validated a novel architecture for univariate TS forecasting.
Revised: We proposed and empirically validated a novel architecture for univariate TS forecasting.

Original: We showed that the architecture is general, ﬂexible and it performs well on a wide array of TS forecasting problems.
Revised: We showed that the architecture is general, ﬂexible and it performs well on a wide array of TS forecasting problems.

Original: We applied it to three non-overlapping challenging competition datasets: M4, M3 and TOURISM and demonstrated state-of-the-art performance in two conﬁgurations: generic and interpretable.
Revised: We applied it to three non-overlapping challenging competition datasets: M4, M3 and TOURISM and demonstrated state-of-the-art performance in two conﬁgurations: generic and interpretable.

Original: This allowed us to validate two important hypotheses: (i) the generic DL approach performs exceptionally well on heterogeneous univariate TS forecasting problems using no TS domain knowledge, (ii) it is viable to additionally constrain a DL model to force it to decompose its forecast into distinct human interpretable outputs.
Revised: This allowed us to validate two important hypotheses: (i) the generic DL approach performs exceptionally well on heterogeneous univariate TS forecasting problems using no TS domain knowledge, (ii) it is viable to additionally constrain a DL model to force it to decompose its forecast into distinct human interpretable outputs.

Original: We also demonstrated that the DL models can be trained on multiple time series in a multi-task fashion, successfully transferring and sharing individual learnings.
Revised: We also demonstrated that the DL models can be trained on multiple time series in a multi-task fashion, successfully transferring and sharing individual learnings.

Original: We speculate that
Revised: We speculate that

Original:  N-BEATS’s performance can be attributed in part to it carrying out a form of meta-learning, a deeper investigation of 
Revised: [Figure] N-BEATS’s performance can be attributed in part to it carrying out a form of meta-learning, a deeper investigation of which should be the subject of future work.

Original: A D ISCUSSION : C ONNECTIONS TO M ETA - LEARNING
Revised: A D ISCUSSION : C ONNECTIONS TO M ETA - LEARNING

Original: Meta-learning deﬁnes an inner learning procedure and an outer learning procedure .
Revised: Meta-learning deﬁnes an inner learning procedure and an outer learning procedure .

Original: The inner learning procedure is parameterized, conditioned or otherwise inﬂuenced by the outer learning procedure (Bengio et al., 1991).
Revised: The inner learning procedure is parameterized, conditioned or otherwise inﬂuenced by the outer learning procedure (Bengio et al., 1991).

Original: The prototypical inner vs. outer learning is individual learning in the lifetime of an animal vs. evolution of the inner learning procedure itself over many generations of individuals.
Revised: The prototypical inner vs. outer learning is individual learning in the lifetime of an animal vs. evolution of the inner learning procedure itself over many generations of individuals.

Original: To see the two levels, it often helps to refer to two sets of parameters, the inner parameters (e.g. synaptic weights) which are modiﬁed inside the inner learning procedure, and the outer parameters or meta-parameters (e.g. genes) which are get modiﬁed only in the outer learning procedure.
Revised: To see the two levels, it often helps to refer to two sets of parameters, the inner parameters (e.g. synaptic weights) which are modiﬁed inside the inner learning procedure, and the outer parameters or meta-parameters (e.g. genes) which are get modiﬁed only in the outer learning procedure.

Original: N-BEATS can be cast as an instance of meta-learning by drawing the following parallels.
Revised: N-BEATS can be cast as an instance of meta-learning by drawing the following parallels.

Original: The outer learning procedure is encapsulated in the parameters of the whole network, learned by gradient descent.
Revised: The outer learning procedure is encapsulated in the parameters of the whole network, learned by gradient descent.

Original: The inner learning procedure is encapsulated in the set of basic building blocks and modiﬁes the parameters θ of g θ .
Revised: The inner learning procedure is encapsulated in the set of basic building blocks and modiﬁes the expansion coefﬁcients θ f that basis g f takes as inputs.

Original: The inner learning proceeds through a sequence of stages, each corresponding to a block within the stack of the architecture.
Revised: The inner learning proceeds through a sequence of stages, each corresponding to a block within the stack of the architecture.

Original: Each of the blocks can be thought of as performing the equivalent of an update step which gradually modiﬁes the parameters θ  which eventually feed into g θ in each block (which get added together to form the ﬁnal prediction).
Revised: Each of the blocks can be thought of as performing the equivalent of an update step which gradually modiﬁes the expansion coefﬁcients θ f which eventually feed into g f in each block (which get added together to form the ﬁnal prediction).

Original: The inner learning procedure takes a single history from a piece of a TS and sees that history as a training set.
Revised: The inner learning procedure takes a single history from a piece of a TS and sees that history as a training set.

Original: It produces forward parameters θ f
Revised: It produces forward expansion coefﬁcients θ f

Original: (see Fig. 1), which parametrically map inputs to predictions.
Revised: (see Fig. 1), which parametrically map inputs to predictions.

Original: In addition, each preceding block modiﬁes the input to the next block by producing backward parameters θ b , thus conditioning the learning and the output of the next block.
Revised: In addition, each preceding block modiﬁes the input to the next block by producing backward expansion coefﬁcients θ b , thus conditioning the learning and the output of the next block.

Original: In the case of the interpretable model, the meta-parameters are only in the FC layers because the g θ ’s are ﬁxed.
Revised: In the case of the interpretable model, the meta-parameters are only in the FC layers because the g f ’s are ﬁxed.

Original: In the case of the generic model, the meta-parameters also include the W ’s which deﬁne the g θ non-parametrically.
Revised: In the case of the generic model, the meta-parameters also include the V ’s which deﬁne the g f non-parametrically.

Original: This point of view is further reinforced by the results of the ablation study reported in Appendix C showing that increasing the number of blocks in the stack, as well as the number of stacks improves generalization performance, and can be interpreted as more iterations of the inner learning procedure.
Revised: This point of view is further reinforced by the results of the ablation study reported in Appendix C showing that increasing the number of blocks in the stack, as well as the number of stacks improves generalization performance, and can be interpreted as more iterations of the inner learning procedure.

Original: B D ATASET D ETAILS B. M4 D ATASET D ETAILS
Revised: B D ATASET D ETAILS B. M4 D ATASET D ETAILS

Original: Table 2 outlines the composition of the M4 dataset across domains and forecast horizons by listing the number of time series based on their frequency and type (M4 Team, 2018b).
Revised: Table 2 outlines the composition of the M4 dataset across domains and forecast horizons by listing the number of time series based on their frequency and type (M4 Team, 2018b).

Original: The M4 dataset is large and diverse: all forecast horizons are composed of heterogeneous time series types (with exception of
Revised: The M4 dataset is large and diverse: all forecast horizons are composed of heterogeneous time series types (with exception of

Original: Hourly) frequently encountered in business, ﬁnancial and economic forecasting.
Revised: Hourly) frequently encountered in business, ﬁnancial and economic forecasting.

Original: Summary statistics on series lengths are also listed, showing wide variability therein, as well as a characterization ( smooth vs erratic ) that follows Syntetos et al.
Revised: Summary statistics on series lengths are also listed, showing wide variability therein, as well as a characterization ( smooth vs erratic ) that follows Syntetos et al.

Original: and is based on the squared coefﬁcient of variation of the series.
Revised: and is based on the squared coefﬁcient of variation of the series.

Original: All series have positive observed values at all time-steps; as such, none can be considered intermittent or lumpy per Syntetos et al.
Revised: All series have positive observed values at all time-steps; as such, none can be considered intermittent or lumpy per Syntetos et al.

Original: B. M3 D ATASET D ETAILS
Revised: B. M3 D ATASET D ETAILS

Original: Table 3 outlines the composition of the M3 dataset across domains and forecast horizons by listing the number of time series based on their frequency and type (Makridakis & Hibon, 2000).
Revised: Table 3 outlines the composition of the M3 dataset across domains and forecast horizons by listing the number of time series based on their frequency and type (Makridakis & Hibon, 2000).

Original: The
Revised: The

Original: M3 is smaller than the M4, but it is still large and diverse: all forecast horizons are composed of heterogeneous time series types frequently encountered in business, ﬁnancial and economic forecasting.
Revised: M3 is smaller than the M4, but it is still large and diverse: all forecast horizons are composed of heterogeneous time series types frequently encountered in business, ﬁnancial and economic forecasting.

Original: Summary statistics on series lengths are also listed, showing wide variability in length, as well as a characterization ( smooth vs erratic ) that follows Syntetos et al.
Revised: Summary statistics on series lengths are also listed, showing wide variability in length, as well as a characterization ( smooth vs erratic ) that follows Syntetos et al.

Original: and is based on the squared coefﬁcient of variation of the series.
Revised: and is based on the squared coefﬁcient of variation of the series.

Original: All series have positive observed values at all time-steps; as such, none can be considered intermittent or lumpy per Syntetos et al.
Revised: All series have positive observed values at all time-steps; as such, none can be considered intermittent or lumpy per Syntetos et al.

Original: B. TOURISM D ATASET D ETAILS
Revised: B. TOURISM D ATASET D ETAILS

Original: Table 4 outlines the composition of the TOURISM dataset across forecast horizons by listing the number of time series based on their frequency.
Revised: Table 4 outlines the composition of the TOURISM dataset across forecast horizons by listing the number of time series based on their frequency.

Original: Summary statistics on series lengths are listed,  showing wide variability in length.
Revised: Summary statistics on series lengths are listed, [Table] [Table] showing wide variability in length.

Original: All series have positive observed values at all time-steps.
Revised: All series have positive observed values at all time-steps.

Original: In contrast to M4 and M3 datasets, TOURISM includes a much higher fraction of erratic series.
Revised: In contrast to M4 and M3 datasets, TOURISM includes a much higher fraction of erratic series.

Original: [Table]
Revised: [Table]

Original: [Table][Table] [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: C A BLATION S TUDIES C. L AYER STACKING AND B ASIS SYNERGY
Revised: C A BLATION S TUDIES C. L AYER STACKING AND B ASIS SYNERGY

Original: We performed an ablation study on the validation set, using s MAPE metric as performance criterion.
Revised: We performed an ablation study on the validation set, using s MAPE metric as performance criterion.

Original: We addressed two speciﬁc questions with this study.
Revised: We addressed two speciﬁc questions with this study.

Original: First, Is stacking layers helpful?
Revised: First, Is stacking layers helpful?

Original: Second, Does the architecture based on the combination of layers with different basis functions results in better performance than the architecture using only one layer type?
Revised: Second, Does the architecture based on the combination of layers with different basis functions results in better performance than the architecture using only one layer type?

Original: Layer stacking.
Revised: Layer stacking.

Original: We start our study with the generic architecture that consists of stacks of one residual block of 5 FC layers each of the form Fig.
Revised: We start our study with the generic architecture that consists of stacks of one residual block of 5 FC layers each of the form Fig.

Original: and we increase the number of stacks.
Revised: and we increase the number of stacks.

Original: Results presented in Table 5 conﬁrm that increasing the number of stacks decreases error and at certain point the gain saturates.
Revised: Results presented in Table 5 conﬁrm that increasing the number of stacks decreases error and at certain point the gain saturates.

Original: We would like to mention that the network having 30 stack of depth 5 is in fact a very deep network of total depth 150 layers.
Revised: We would like to mention that the network having 30 stack of depth 5 is in fact a very deep network of total depth 150 layers.

Original: Basis synergy.
Revised: Basis synergy.

Original: Stacking works well for the interpretable architecture as can be seen in Tabledepicting the results of ablating the interpretable architecture conﬁguration.
Revised: Stacking works well for the interpretable architecture as can be seen in Tabledepicting the results of ablating the interpretable architecture conﬁguration.

Original: Here we experiment with the architecture that is composed of 2 stacks, stack one is trend model and stack two is the seasonality model.
Revised: Here we experiment with the architecture that is composed of 2 stacks, stack one is trend model and stack two is the seasonality model.

Original: Each stack has variable number of residual blocks and each residual block has
Revised: Each stack has variable number of residual blocks and each residual block has

Original: FC layers.
Revised: FC layers.

Original: We found that this architecture works best when all weights are shared within stack.
Revised: We found that this architecture works best when all weights are shared within stack.

Original: We clearly see that increasing the number of layers improves performance.
Revised: We clearly see that increasing the number of layers improves performance.

Original: The largest network islayers deep.
Revised: The largest network islayers deep.

Original: On top of that, we observe that the architecture that consists of stacks based on different basis functions wins over the architecture based on the same stack.
Revised: On top of that, we observe that the architecture that consists of stacks based on different basis functions wins over the architecture based on the same stack.

Original: It looks like chaining stacks of different nature results in synergistic effects.
Revised: It looks like chaining stacks of different nature results in synergistic effects.

Original: This is logical as function classes that can be modelled by trend and seasonality stacks have small overlap.
Revised: This is logical as function classes that can be modelled by trend and seasonality stacks have small overlap.

Original: C. E NSEMBLE SIZE
Revised: C. E NSEMBLE SIZE

Original: Figure 3 demonstrates that increasing the ensemble size results in improved performance.
Revised: Figure 3 demonstrates that increasing the ensemble size results in improved performance.

Original: Most importantly, according to Figure 3, N-BEATS achieves state-of-the-art performance even if comparatively small ensemble size of 18 models is used.
Revised: Most importantly, according to Figure 3, N-BEATS achieves state-of-the-art performance even if comparatively small ensemble size of 18 models is used.

Original: Therefore, computational efﬁciency of N-BEATS can be traded very effectively for performance and there is no over-reliance of the results on large ensemble size.
Revised: Therefore, computational efﬁciency of N-BEATS can be traded very effectively for performance and there is no over-reliance of the results on large ensemble size.

Original: C. D OUBLY RESIDUAL STACKING
Revised: C. D OUBLY RESIDUAL STACKING

Original: In Section 3.2 we described the proposed doubly residual stacking (DRESS) principle, which is the topological foundation of N-BEATS.
Revised: In Section 3.2 we described the proposed doubly residual stacking (DRESS) principle, which is the topological foundation of N-BEATS.

Original: The topology is based on both (i) running a residual backcast connection and (ii) producing partial block-level forecasts that are further aggregated at stack and model levels to produce the ﬁnal model-level forecast.
Revised: The topology is based on both (i) running a residual backcast connection and (ii) producing partial block-level forecasts that are further aggregated at stack and model levels to produce the ﬁnal model-level forecast.

Original: In this section we conduct a study to conﬁrm the accuracy effectiveness of this topology compared to several alternatives.
Revised: In this section we conduct a study to conﬁrm the accuracy effectiveness of this topology compared to several alternatives.

Original: The methodology underlying this study is that we remove either the backcast or partial forecast links or both and track how this affects the forecasting metrics.
Revised: The methodology underlying this study is that we remove either the backcast or partial forecast links or both and track how this affects the forecasting metrics.

Original: We keep the number of parameters in the network for each of the architectural alternatives ﬁxed by using the same number of layers in the network (we used default hyperparameter settings reported in Table 16).
Revised: We keep the number of parameters in the network for each of the architectural alternatives ﬁxed by using the same number of layers in the network (we used default hyperparameter settings reported in Table 16).

Original: The architectural alternatives are depicted in
Revised: The architectural alternatives are depicted in

Original: Figure 4 and described in detail below.
Revised: Figure 4 and described in detail below.

Original: [Figure]
Revised: [Figure]

Original: N-BEATS-DRESS is depicted in Fig. 4a.
Revised: N-BEATS-DRESS is depicted in Fig. 4a.

Original: This is the default conﬁguration of N-BEATS using doubly residual stacking described in Section 3.2.
Revised: This is the default conﬁguration of N-BEATS using doubly residual stacking described in Section 3.2.

Original: PARALLEL is depicted in Fig.
Revised: PARALLEL is depicted in Fig.

Original: 4b.
Revised: 4b.

Original: This is the alternative where the backward residual connection is disabled and the overall model input is fed to every block.
Revised: This is the alternative where the backward residual connection is disabled and the overall model input is fed to every block.

Original: The blocks then forecast in parallel using the same input and their individual outputs are summed to make the ﬁnal forecast.
Revised: The blocks then forecast in parallel using the same input and their individual outputs are summed to make the ﬁnal forecast.

Original: NO-RESIDUAL is depicted in Fig.
Revised: NO-RESIDUAL is depicted in Fig.

Original: 4c.
Revised: 4c.

Original: This is the alternative where the backward residual connection is disabled.
Revised: This is the alternative where the backward residual connection is disabled.

Original: Unlike PARALLEL, in this case the backcast forecast of the previous block is fed as input to the next block.
Revised: Unlike PARALLEL, in this case the backcast forecast of the previous block is fed as input to the next block.

Original: Unlike the usual feed-forward network, in the NO-RESIDUAL architecture, each block makes a partial forecast and their individual outputs are summed to make the ﬁnal forecast.
Revised: Unlike the usual feed-forward network, in the NO-RESIDUAL architecture, each block makes a partial forecast and their individual outputs are summed to make the ﬁnal forecast.

Original: LAST-FORWARD is depicted in Fig.
Revised: LAST-FORWARD is depicted in Fig.

Original: 4d.
Revised: 4d.

Original: This is the alternative where the backward residual connection is active, however the model level forecast is derived only from the last block.
Revised: This is the alternative where the backward residual connection is active, however the model level forecast is derived only from the last block.

Original: So, the partial forward forecasts are disabled.
Revised: So, the partial forward forecasts are disabled.

Original: This is the architecture that is closest to the classical residual network.
Revised: This is the architecture that is closest to the classical residual network.

Original: NO-RESIDUAL-LAST-FORWARD is depicted in Fig.
Revised: NO-RESIDUAL-LAST-FORWARD is depicted in Fig.

Original: 4e.
Revised: 4e.

Original: This is the alternative where both backward residual and the partial forward connections are disabled.
Revised: This is the alternative where both backward residual and the partial forward connections are disabled.

Original: This is therefore a simple feed-forward network, but very deep.
Revised: This is therefore a simple feed-forward network, but very deep.

Original: The quantitative ablation study results on the M4 dataset are reported in Tables 7–10.
Revised: The quantitative ablation study results on the M4 dataset are reported in Tables 7–10.

Original: N-BEATSDRESS model is essentially N-BEATS model in this study.
Revised: N-BEATSDRESS model is essentially N-BEATS model in this study.

Original: For this study we used ensemble size of 18.
Revised: For this study we used ensemble size of 18.

Original: Since the ensemble size is 18 for N-BEATS-DRESS, as opposed to 180 used for N-BEATS, the OWA metric reported in Table 9 for N-BEATS-DRESS is higher than the OWA reported for
Revised: Since the ensemble size is 18 for N-BEATS-DRESS, as opposed to 180 used for N-BEATS, the OWA metric reported in Table 9 for N-BEATS-DRESS is higher than the OWA reported for

Original: N-BEATS-G in Table 12.
Revised: N-BEATS-G in Table 12.

Original: Note that both results align well with OWA reported in Figure 3 for different ensemble sizes, as part of the ensemble size ablation conducted in Section C.2.
Revised: Note that both results align well with OWA reported in Figure 3 for different ensemble sizes, as part of the ensemble size ablation conducted in Section C.2.

Original: The results presented in Tables 7–10 demonstrate that the doubly residual stacking topology provides a clear overall advantage over the alternative architectures in which either backcast residual links or the partial forward forecast links are disabled.
Revised: The results presented in Tables 7–10 demonstrate that the doubly residual stacking topology provides a clear overall advantage over the alternative architectures in which either backcast residual links or the partial forward forecast links are disabled.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: PARALLEL
Revised: PARALLEL

Original: [Table]
Revised: [Table]

Original: NO
Revised: NO

Original: [Table]
Revised: [Table]

Original: RESIDUAL
Revised: RESIDUAL

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: LAST
Revised: LAST

Original: [Table]
Revised: [Table]

Original: NO-RESIDUAL-LAST-FORWARD-G 15.
Revised: NO-RESIDUAL-LAST-FORWARD-G 15.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: N-BEATS-DRESS-G 13.
Revised: N-BEATS-DRESS-G 13.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: D D ETAILED E MPIRICAL R ESULTS D. D ETAILED RESULTS : M4 D ATASET
Revised: D D ETAILED E MPIRICAL R ESULTS D. D ETAILED RESULTS : M4 D ATASET

Original: Tables 11 and 12 present our key quantitative empirical results showing that the proposed model achieves the state of the art performance on the challenging M4 benchmark.
Revised: Tables 11 and 12 present our key quantitative empirical results showing that the proposed model achieves the state of the art performance on the challenging M4 benchmark.

Original: We study the performance of two model conﬁgurations: generic (Ours-G) and interpretable (Ours-I), as well as Ours-I+G (ensemble of all models from Ours-G and Ours-I).
Revised: We study the performance of two model conﬁgurations: generic (Ours-G) and interpretable (Ours-I), as well as Ours-I+G (ensemble of all models from Ours-G and Ours-I).

Original: We compare against 4 representatives from the M competition: each best in their respective model class.
Revised: We compare against 4 representatives from the M competition: each best in their respective model class.

Original: Best pure ML is the submission by B. Trotta, the best entry among the 6 pure ML models.
Revised: Best pure ML is the submission by B. Trotta, the best entry among the 6 pure ML models.

Original: Best statistical is the best pure statistical model by N.Z.
Revised: Best statistical is the best pure statistical model by N.Z.

Original: Legaki and K. Koutsouri.
Revised: Legaki and K. Koutsouri.

Original: Best ML/TS combination is the model by P. Montero-Manso, T. Talagala,
Revised: Best ML/TS combination is the model by P. Montero-Manso, T. Talagala,

Original: R.J. Hyndman and G. Athanasopoulos, second best entry, gradient boosted tree over a few statistical time series models.
Revised: R.J. Hyndman and G. Athanasopoulos, second best entry, gradient boosted tree over a few statistical time series models.

Original: Finally, DL/TS hybrid is the winner of M4 competition designed by S. Smyl.
Revised: Finally, DL/TS hybrid is the winner of M4 competition designed by S. Smyl.

Original: N-BEATS outperforms all other approaches on all the studied subsets of time series.
Revised: N-BEATS outperforms all other approaches on all the studied subsets of time series.

Original: The average
Revised: The average

Original: OWA gap between our generic model and the M4 winner (0 . 821 − 0 .
Revised: OWA gap between our generic model and the M4 winner (0 . 821 − 0 .

Original: 795 = 0 . 026) is greater than the gap between the M4 winner and the second entry (0 . 838 − 0 .
Revised: 795 = 0 . 026) is greater than the gap between the M4 winner and the second entry (0 . 838 − 0 .

Original: = 0 . 017).
Revised: = 0 . 017).

Original: A more granular and detailed statistical analysis of our results on M4 is provided in Table 13.
Revised: A more granular and detailed statistical analysis of our results on M4 is provided in Table 13.

Original: This table ﬁrst presents the s MAPE for N-BEATS, decomposed by M4 time series sub-type and sampling frequency (upper part).
Revised: This table ﬁrst presents the s MAPE for N-BEATS, decomposed by M4 time series sub-type and sampling frequency (upper part).

Original: Then (lower part), it shows the average s MAPE difference between the
Revised: Then (lower part), it shows the average s MAPE difference between the

Original: N-BEATS results and the M4 winner (TS/DL hybrid by S. Smyl), adding the standard error of that difference (in parentheses)
Revised: N-BEATS results and the M4 winner (TS/DL hybrid by S. Smyl), adding the standard error of that difference (in parentheses)

Original: ; bold entries indicate statistical signiﬁcance at the 99% level based on a two-sided paired t -test.
Revised: ; bold entries indicate statistical signiﬁcance at the 99% level based on a two-sided paired t -test.

Original: We note that each cross-section of the M4 dataset into horizon and type may be regarded as an independent mini-dataset.
Revised: We note that each cross-section of the M4 dataset into horizon and type may be regarded as an independent mini-dataset.

Original: We observe that over those mini-datasets there is a preponderance of statistically signiﬁcant differences between N-BEATS and Smyl (18 cases out of 31) to the advantage of N-BEATS.
Revised: We observe that over those mini-datasets there is a preponderance of statistically signiﬁcant differences between N-BEATS and Smyl (18 cases out of 31) to the advantage of N-BEATS.

Original: This provides evidence that (i) the improvement observed on average in Tablesand 12 is statistically signiﬁcant and consistent over smaller subsets of M4 and (ii) N-BEATS generalizes well over time series of different types and sampling frequencies.
Revised: This provides evidence that (i) the improvement observed on average in Tablesand 12 is statistically signiﬁcant and consistent over smaller subsets of M4 and (ii) N-BEATS generalizes well over time series of different types and sampling frequencies.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: D ETAILED RESULTS : M3 D ATASET
Revised: D ETAILED RESULTS : M3 D ATASET

Original: Results for M3 dataset are provided in Table 14.
Revised: Results for M3 dataset are provided in Table 14.

Original: The performance metric is calculated using the earlier version of s MAPE , deﬁned speciﬁcally for the M3 competition:
Revised: The performance metric is calculated using the earlier version of s MAPE , deﬁned speciﬁcally for the M3 competition:

Original: [Equation]
Revised: [Equation]

Original: For some of the methods, either average s MAPE was not reported or s MAPE for some of the splits was not reported in their respective publications.
Revised: For some of the methods, either average s MAPE was not reported or s MAPE for some of the splits was not reported in their respective publications.

Original: Below, we list those cases.
Revised: Below, we list those cases.

Original: BaggedETS.BC (Bergmeir et al., 2016) has not reported numbers on Others.
Revised: BaggedETS.BC (Bergmeir et al., 2016) has not reported numbers on Others.

Original: LGT (Smyl & Kuber, 2016) did not report results on
Revised: LGT (Smyl & Kuber, 2016) did not report results on

Original: Monthly and Quarterly data.
Revised: Monthly and Quarterly data.

Original: According to the authors, the underlying RNN had problems dealing with raw seasonal data, the ETS based pre-processing was not effective and the LGT pre-processing was not computationally feasible given comparatively large number of time series and their comparatively large length (Smyl & Kuber, 2016).
Revised: According to the authors, the underlying RNN had problems dealing with raw seasonal data, the ETS based pre-processing was not effective and the LGT pre-processing was not computationally feasible given comparatively large number of time series and their comparatively large length (Smyl & Kuber, 2016).

Original: Finally, EXP (Spiliotis et al., 2019) reported average performance computed using a different methodology than the default M3 and M4 methodology
Revised: Finally, EXP (Spiliotis et al., 2019) reported average performance computed using a different methodology than the default M3 and M4 methodology

Original: (source: personal communication with the authors).
Revised: (source: personal communication with the authors).

Original: For the latter method we recomputed the Average s MAPE based on the previously reported Yearly, Quarterly and Monthly splits.
Revised: For the latter method we recomputed the Average s MAPE based on the previously reported Yearly, Quarterly and Monthly splits.

Original: To calculate it, we follow the M3, M and TOURISM competition methodology and compute the average metric as the average over all time series and over all forecast horizons.
Revised: To calculate it, we follow the M3, M and TOURISM competition methodology and compute the average metric as the average over all time series and over all forecast horizons.

Original: Given the performance metric values aggregated over Yearly,
Revised: Given the performance metric values aggregated over Yearly,

Original: Quarterly and Monthly splits, the average can be computed straightforwardly as:
Revised: Quarterly and Monthly splits, the average can be computed straightforwardly as:

Original: [Equation]
Revised: [Equation]

Original: Here N Tot = N Year + N Quart + N Month + N Others and N Year = 6 × 645 , N Quart = 8 × 756 , N Month = 18 × 1428 , N Others = 8 × 174.
Revised: Here N Tot = N Year + N Quart + N Month + N Others and N Year = 6 × 645 , N Quart = 8 × 756 , N Month = 18 × 1428 , N Others = 8 × 174.

Original: It is clear that for each split, its N is the product of its respective number of time series and its largest forecast horizon.
Revised: It is clear that for each split, its N is the product of its respective number of time series and its largest forecast horizon.

Original: [Table]
Revised: [Table]

Original: D. D ETAILED RESULTS : TOURISM D ATASET
Revised: D. D ETAILED RESULTS : TOURISM D ATASET

Original: Detailed results for the TOURISM competition dataset are provided in Table 15.
Revised: Detailed results for the TOURISM competition dataset are provided in Table 15.

Original: The respective Kaggle competition was divided into two parts: (i) Yearly time series forecasting and (ii) Quarterly/Monthly time series forecasting (Athanasopoulos & Hyndman, 2011).
Revised: The respective Kaggle competition was divided into two parts: (i) Yearly time series forecasting and (ii) Quarterly/Monthly time series forecasting (Athanasopoulos & Hyndman, 2011).

Original: Some of the participants chose to take part only in the second part.
Revised: Some of the participants chose to take part only in the second part.

Original: Therefore, In addition to entries present in Table 1, we report competitors from (Athanasopoulos & Hyndman, 2011) that have missing results in Yearly competition.
Revised: Therefore, In addition to entries present in Table 1, we report competitors from (Athanasopoulos & Hyndman, 2011) that have missing results in Yearly competition.

Original: In particular, SaliMali team is the winner of the Quarterly/Monthly time series forecasting competition (Brierley, 2011).
Revised: In particular, SaliMali team is the winner of the Quarterly/Monthly time series forecasting competition (Brierley, 2011).

Original: Their approach is based on a weighted ensemble of statistical methods.
Revised: Their approach is based on a weighted ensemble of statistical methods.

Original: Teams Robert and Idalgo used unknown approaches.
Revised: Teams Robert and Idalgo used unknown approaches.

Original: We can see from Table 15 that N-BEATS achieves state-of-the-art performance on all subsets of TOURISM dataset.
Revised: We can see from Table 15 that N-BEATS achieves state-of-the-art performance on all subsets of TOURISM dataset.

Original: On average, it is state of the art and it gains 4.2% over the best-known approach LeeCBaker , and 11.5% over auto-ARIMA.
Revised: On average, it is state of the art and it gains 4.2% over the best-known approach LeeCBaker , and 11.5% over auto-ARIMA.

Original: The average metrics have not been reported in the original competition results (Athanasopoulos et al., 2011; Athanasopoulos & Hyndman, 2011).
Revised: The average metrics have not been reported in the original competition results (Athanasopoulos et al., 2011; Athanasopoulos & Hyndman, 2011).

Original: Therefore, in Table 15, we present the Average MAPE metric calculated by us based on the previously reported Yearly, Quarterly and Monthly splits.
Revised: Therefore, in Table 15, we present the Average MAPE metric calculated by us based on the previously reported Yearly, Quarterly and Monthly splits.

Original: To calculate it, we follow the M4 competition methodology and compute the average metric as the average over all time series and over all forecast horizons.
Revised: To calculate it, we follow the M4 competition methodology and compute the average metric as the average over all time series and over all forecast horizons.

Original: Given the performance metric values aggregated over Yearly, Quarterly and Monthly splits, the average can be computed straightforwardly as:
Revised: Given the performance metric values aggregated over Yearly, Quarterly and Monthly splits, the average can be computed straightforwardly as:

Original: [Equation]
Revised: [Equation]

Original: Here N Tot = N Year + N Quart + N Month and N Year = 4 × 518 , N Quart = 8 × 427 , N Month = 24 × 366.
Revised: Here N Tot = N Year + N Quart + N Month and N Year = 4 × 518 , N Quart = 8 × 427 , N Month = 24 × 366.

Original: It is clear that for each split, its N is the product of its respective number of time series and its largest forecast horizon.
Revised: It is clear that for each split, its N is the product of its respective number of time series and its largest forecast horizon.

Original: [Table]
Revised: [Table]

Original: E H YPER - PARAMETER SETTINGS
Revised: E H YPER - PARAMETER SETTINGS

Original: Table 16 presents the hyperparameter settings used to train models on different subsets of M4, M and TOURISM datasets.
Revised: Table 16 presents the hyperparameter settings used to train models on different subsets of M4, M and TOURISM datasets.

Original: A brief discussion of ﬁeld names in the table is warranted.
Revised: A brief discussion of ﬁeld names in the table is warranted.

Original: Subset names Yly, Qly, Mly, Wly, Dly, Hly, Other correspond to yearly, quarterly, monthly, weekly, daily, hourly and other frequency subsets deﬁned in the original datasets.
Revised: Subset names Yly, Qly, Mly, Wly, Dly, Hly, Other correspond to yearly, quarterly, monthly, weekly, daily, hourly and other frequency subsets deﬁned in the original datasets.

Original: N-BEATS-I and N-BEATS-G correspond to the interpretable and generic model conﬁgurations deﬁned in Section 3.3.
Revised: N-BEATS-I and N-BEATS-G correspond to the interpretable and generic model conﬁgurations deﬁned in Section 3.3.

Original: E. C OMMON PARAMETERS
Revised: E. C OMMON PARAMETERS

Original: L H is the coefﬁcient deﬁning the length of training history immediately preceding the last point in the train part of the TS that is used to generate training samples.
Revised: L H is the coefﬁcient deﬁning the length of training history immediately preceding the last point in the train part of the TS that is used to generate training samples.

Original: For example, if for M4 Yearly the forecast horizon is 6 and L H is 1.5, then we consider 1 . 5 · 6 = 9 most recent points in the train dataset for each time series to generate training samples.
Revised: For example, if for M4 Yearly the forecast horizon is 6 and L H is 1.5, then we consider 1 . 5 · 6 = 9 most recent points in the train dataset for each time series to generate training samples.

Original: A training sample from a given TS in M4 Yearly is then generated by choosing one of the most recent 9 points as an anchor.
Revised: A training sample from a given TS in M4 Yearly is then generated by choosing one of the most recent 9 points as an anchor.

Original: All the points preceding the anchor are used to create the input to N-BEATS, while the points following and including the anchor become training target.
Revised: All the points preceding the anchor are used to create the input to N-BEATS, while the points following and including the anchor become training target.

Original: Target and history points that fall outside of the time series limits given the anchor position are ﬁlled with zeros and masked during the training.
Revised: Target and history points that fall outside of the time series limits given the anchor position are ﬁlled with zeros and masked during the training.

Original: We observed that for subsets with large number of time series L H tends to be smaller and for subsets with smaller number of time series it tends to be larger.
Revised: We observed that for subsets with large number of time series L H tends to be smaller and for subsets with smaller number of time series it tends to be larger.

Original: For example, in massive Yearly, Monthly, Quarterly subsets of M4 L H is equal to 1 . 5; and in moderate to small Weekly, Daily, Hourly subsets of M4 L H is equal to 10.
Revised: For example, in massive Yearly, Monthly, Quarterly subsets of M4 L H is equal to 1 . 5; and in moderate to small Weekly, Daily, Hourly subsets of M4 L H is equal to 10.

Original: Iterations is the number of batches used to train N-BEATS.
Revised: Iterations is the number of batches used to train N-BEATS.

Original: Losses is the set of loss functions that is used to build ensemble.
Revised: Losses is the set of loss functions that is used to build ensemble.

Original: We observed on the respective validation sets that for M4 and M3 mixing models trained on a variety of metrics resulted in performance gain.
Revised: We observed on the respective validation sets that for M4 and M3 mixing models trained on a variety of metrics resulted in performance gain.

Original: In the case of TOURISM dataset training only on MAPE led to the best validation scores.
Revised: In the case of TOURISM dataset training only on MAPE led to the best validation scores.

Original: Sharing deﬁnes whether the coefﬁcients in the fully-connected layers are shared.
Revised: Sharing deﬁnes whether the coefﬁcients in the fully-connected layers are shared.

Original: We observed that the interpretable model works best when weights are shared across stack, while generic model works best when none of the weights are shared.
Revised: We observed that the interpretable model works best when weights are shared across stack, while generic model works best when none of the weights are shared.

Original: Lookback period is the length of the history window forming the input to the model (please refer to
Revised: Lookback period is the length of the history window forming the input to the model (please refer to

Original: Figure 1).
Revised: Figure 1).

Original: This is the function of the forecast horizon length, H .
Revised: This is the function of the forecast horizon length, H .

Original: In our experiments we mixed models with lookback periods 2 H , 3 H , 4 H , 5 H , 6 H , 7 H in one ensemble.
Revised: In our experiments we mixed models with lookback periods 2 H , 3 H , 4 H , 5 H , 6 H , 7 H in one ensemble.

Original: As an example, for a forecast horizon length H = 8 and a lookback period 7 H , the model’s input will consist of the history window of 7 · 8 = 56 samples.
Revised: As an example, for a forecast horizon length H = 8 and a lookback period 7 H , the model’s input will consist of the history window of 7 · 8 = 56 samples.

Original: Batch is the batch size.
Revised: Batch is the batch size.

Original: We used batch size of 1024.
Revised: We used batch size of 1024.

Original: We observed that the training was faster with larger batch sizes, however in our setup little gain was observed with batch sizes beyond 1024.
Revised: We observed that the training was faster with larger batch sizes, however in our setup little gain was observed with batch sizes beyond 1024.

Original: E. N-BEATS-I PARAMETERS
Revised: E. N-BEATS-I PARAMETERS

Original: S-width is the width of the fully connected layers in the blocks comprising the seasonality stack of the interpretable model (please refer to Figure 1).
Revised: S-width is the width of the fully connected layers in the blocks comprising the seasonality stack of the interpretable model (please refer to Figure 1).

Original: S-blocks is the number of blocks comprising the seasonality stack of the interpretable model (please refer to Figure 1).
Revised: S-blocks is the number of blocks comprising the seasonality stack of the interpretable model (please refer to Figure 1).

Original: S-block-layers is the number of fully-connected layers comprising one block in the seasonality stack of the interpretable model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1).
Revised: S-block-layers is the number of fully-connected layers comprising one block in the seasonality stack of the interpretable model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1).

Original: T-width is the width of the fully connected layers in the blocks comprising the trend stack of the interpretable model (please refer to Figure 1).
Revised: T-width is the width of the fully connected layers in the blocks comprising the trend stack of the interpretable model (please refer to Figure 1).

Original: T-degree is the degree p of polynomial in the trend stack of the interpretable model (please refer to equation (1)).
Revised: T-degree is the degree p of polynomial in the trend stack of the interpretable model (please refer to equation (2)).

Original: T-blocks is the number of blocks comprising the trend stack of the interpretable model (please refer to Figure 1).
Revised: T-blocks is the number of blocks comprising the trend stack of the interpretable model (please refer to Figure 1).

Original: T-block-layers is the number of fully-connected
Revised: T-block-layers is the number of fully-connected

Original: layers comprising one block in the trend stack of the interpretable model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1).
Revised: layers comprising one block in the trend stack of the interpretable model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1).

Original: E. N-BEATS-G PARAMETERS
Revised: E. N-BEATS-G PARAMETERS

Original: Width is the width of the fully connected layers in the blocks comprising the stacks of the generic model (please refer to Figure 1).
Revised: Width is the width of the fully connected layers in the blocks comprising the stacks of the generic model (please refer to Figure 1).

Original: Blocks is the number of blocks comprising the stack of the generic model (please refer to Figure 1). 
Revised: Blocks is the number of blocks comprising the stack of the generic model (please refer to Figure 1). Block-layers is the number of fully-connected layers comprising one block in the stack of the generic model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1). [Figure] F D ETAILED SIGNAL TRACES OF INTERPRETABLE INPUTS PRESENTED IN F IGURE

Revised: The goal of this section is to show the detailed traces (numeric values) of signals visualized in Fig.

Original: Block-layers is  the number of fully-connected layers comprising one block in the stack of the generic model (preceding the ﬁnal fully-connected projection layers forming the backcast/forecast fork, please refer to Figure 1).
Revised: This is to demonstrate that even though the StackT-I (Fig. 2 (d)) and StackS-I (Fig. 2 (e)) provide response lines different from the counterparts in Stack1-G (Fig. 2 (b)) and Stack2-G (Fig. 2 (c)), the summations in the combined line (Fig. 2 (a)) can still be very similar. First, we reproduce Fig.

Revised: 5 for the convenience of the reader.

Revised: Second, for each row in the ﬁgure, we produce a table showing the numeric values of each signal depicted in corresponding plots (please refer to Tables 17– 22).

Revised: We make sure that the names of signals in ﬁgure legends and in the table columns match, such that they can easily be cross-referenced.

Revised: It can be clearly seen in Tables 17–that (i) traces STACK1-I and STACK2-I sum up to trace FORECAST-I, (ii) traces STACK1-G and

Revised: STACK2-G sum up to trace FORECAST-G, (iii) traces FORECAST-I and FORECAST-G are overall very similar even though their components may signiﬁcantly differ from each other.

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]

Revised: [Table]