N-BEATS: Neural basis expansion analysis for interpretable time series forecasting

We focus on solving the univariate times series point forecasting problem using deep learning . We propose a deep neural architecture based on backward and forward residual links and a very deep stack of fully-connected layers . The architecture has a number of desirable properties , being interpretable , applicable without modification to a wide array of target domains , and fast to train . We test the proposed architecture on several well-known datasets , including M3 , M4 and TOURISM competition datasets containing time series from diverse domains . We demonstrate state-of-the-art performance for two configurations of N-BEATS for all the datasets , improving forecast accuracy by 11 % over a statistical benchmark and by 3 % over last year ’ s winner of the M4 competition , a domain-adjusted hand-crafted hybrid between neural network and statistical time series models . The first configuration of our model does not employ any time-series-specific components and its performance on heterogeneous datasets strongly suggests that , contrarily to received wisdom , deep learning primitives such as residual blocks are by themselves sufficient to solve a wide range of forecasting problems . Finally , we demonstrate how the proposed architecture can be augmented to provide outputs that are interpretable without considerable loss in accuracy . 1 INTRODUCTION . Time series ( TS ) forecasting is an important business problem and a fruitful application area for machine learning ( ML ) . It underlies most aspects of modern business , including such critical areas as inventory control and customer management , as well as business planning going from production and distribution to finance and marketing . As such , it has a considerable financial impact , often ranging in the millions of dollars for every point of forecasting accuracy gained ( Jain , 2017 ; Kahn , 2003 ) . And yet , unlike areas such as computer vision or natural language processing where deep learning ( DL ) techniques are now well entrenched , there still exists evidence that ML and DL struggle to outperform classical statistical TS forecasting approaches ( Makridakis et al. , 2018a ; b ) . For instance , the rankings of the six “ pure ” ML methods submitted to M4 competition were 23 , 37 , 38 , 48 , 54 , and 57 out of a total of 60 entries , and most of the best-ranking methods were ensembles of classical statistical techniques ( Makridakis et al. , 2018b ) . On the other hand , the M4 competition winner ( Smyl , 2020 ) , was based on a hybrid between neural residual/attention dilated LSTM stack with a classical Holt-Winters statistical model ( Holt , 1957 ; 2004 ; Winters , 1960 ) with learnable parameters . Since Smyl ’ s approach heavily depends on this Holt-Winters component , Makridakis et al . ( 2018b ) further argue that “ hybrid approaches and combinations of method are the way forward for improving the forecasting accuracy and making forecasting more valuable ” . In this work we aspire to challenge this conclusion by exploring the potential of pure DL architectures in the context of the TS forecasting . Moreover , in the context of interpretable DL architecture design , we are interested in answering the following question : can we inject a suitable inductive bias in the model to make its internal operations more interpretable , in the sense of extracting some explainable driving factors combining to produce a given forecast ? 1.1 SUMMARY OF CONTRIBUTIONS . Deep Neural Architecture : To the best of our knowledge , this is the first work to empirically demonstrate that pure DL using no time-series specific components outperforms well-established statistical approaches on M3 , M4 and TOURISM datasets ( on M4 , by 11 % over statistical benchmark , by 7 % over the best statistical entry , and by 3 % over the M4 competition winner ) . In our view , this provides a long-missing proof of concept for the use of pure ML in TS forecasting and strengthens motivation to continue advancing the research in this area . Interpretable DL for Time Series : In addition to accuracy benefits , we also show that it is feasible to design an architecture with interpretable outputs that can be used by practitioners in very much the same way as traditional decomposition techniques such as the “ seasonality-trend-level ” approach ( Cleveland et al. , 1990 ) . 2 PROBLEM STATEMENT . We consider the univariate point forecasting problem in discrete time . Given a length-H forecast horizon a length-T observed series history [ y1 , . . . , yT ] ∈ RT , the task is to predict the vector of future values y ∈ RH = [ yT+1 , yT+2 , . . . , yT+H ] . For simplicity , we will later consider a lookback window of length t ≤ T ending with the last observed value yT to serve as model input , and denoted x ∈ Rt = [ yT−t+1 , . . . , yT ] . We denote ŷ the forecast of y . The following metrics are commonly used to evaluate forecasting performance ( Hyndman & Koehler , 2006 ; Makridakis & Hibon , 2000 ; Makridakis et al. , 2018b ; Athanasopoulos et al. , 2011 ) : sMAPE = 200 H H ∑ i=1 |yT+i− ŷT+i| |yT+i|+ |ŷT+i| , MAPE = 100 H H ∑ i=1 |yT+i− ŷT+i| |yT+i| , MASE = 1 H H ∑ i=1 |yT+i− ŷT+i| 1 T+H−m ∑ T+H j=m+1 |y j− y j−m| , OWA = 1 2 [ sMAPE sMAPENaïve2 + MASE MASENaïve2 ] . Here m is the periodicity of the data ( e.g. , 12 for monthly series ) . MAPE ( Mean Absolute Percentage Error ) , sMAPE ( symmetric MAPE ) and MASE ( Mean Absolute Scaled Error ) are standard scale-free metrics in the practice of forecasting ( Hyndman & Koehler , 2006 ; Makridakis & Hibon , 2000 ) : whereas sMAPE scales the error by the average between the forecast and ground truth , the MASE scales by the average error of the naïve predictor that simply copies the observation measured m periods in the past , thereby accounting for seasonality . OWA ( overall weighted average ) is a M4specific metric used to rank competition entries ( M4 Team , 2018b ) , where sMAPE and MASE metrics are normalized such that a seasonally-adjusted naïve forecast obtains OWA = 1.0 . 3 N-BEATS . Our architecture design methodology relies on a few key principles . First , the base architecture should be simple and generic , yet expressive ( deep ) . Second , the architecture should not rely on timeseries-specific feature engineering or input scaling . These prerequisites let us explore the potential of pure DL architecture in TS forecasting . Finally , as a prerequisite to explore interpretability , the architecture should be extendable towards making its outputs human interpretable . We now discuss how those principles converge to the proposed architecture . 3.1 BASIC BLOCK . The proposed basic building block has a fork architecture and is depicted in Fig . 1 ( left ) . We focus on describing the operation of ` -th block in this section in detail ( note that the block index ` is dropped in Fig . 1 for brevity ) . The ` -th block accepts its respective input x ` and outputs two vectors , x̂ ` and ŷ ` . For the very first block in the model , its respective x ` is the overall model input — a history lookback window of certain length ending with the last measured observation . We set the length of input window to a multiple of the forecast horizon H , and typical lengths of x in our setup range from 2H to 7H . For the rest of the blocks , their inputs x ` are residual outputs of the previous blocks . Each block has two outputs : ŷ ` , the block ’ s forward forecast of length H ; and x̂ ` , the block ’ s best estimate of x ` , also known as the ‘ backcast ’ , given the constraints on the functional space that the block can use to approximate signals . Internally , the basic building block consists of two parts . The first part is a fully connected network that produces the forward θ f ` and the backward θ b ` predictors of expansion coefficients ( again , note that the block index ` is dropped for θ b ` , θ f ` , g b ` , g f ` in Fig . 1 for brevity ) . The second part consists of the backward gb ` and the forward g f ` basis layers that accept the respective forward θ f ` and backward θ b ` expansion coefficients , project them internally on the set of basis functions and produce the backcast x̂ ` and the forecast outputs ŷ ` defined in the previous paragraph . The operation of the first part of the ` -th block is described by the following equations : h ` ,1 = FC ` ,1 ( x ` ) , h ` ,2 = FC ` ,2 ( h ` ,1 ) , h ` ,3 = FC ` ,3 ( h ` ,2 ) , h ` ,4 = FC ` ,4 ( h ` ,3 ) . θ b ` = LINEAR b ` ( h ` ,4 ) , θ f ` = LINEAR f ` ( h ` ,4 ) . ( 1 ) Here LINEAR layer is simply a linear projection layer , i.e . θ f ` = W f ` h ` ,4 . The FC layer is a standard fully connected layer with RELU non-linearity ( Nair & Hinton , 2010 ) , such that for FC ` ,1 we have , for example : h ` ,1 = RELU ( W ` ,1x ` +b ` ,1 ) . One task of this part of the architecture is to predict the forward expansion coefficients θ f ` with the ultimate goal of optimizing the accuracy of the partial forecast ŷ ` by properly mixing the basis vectors supplied by g f ` . Additionally , this sub-network predicts backward expansion coefficients θ b ` used by g b ` to produce an estimate of x ` with the ultimate goal of helping the downstream blocks by removing components of their input that are not helpful for forecasting . The second part of the network maps expansion coefficients θ f ` and θ b ` to outputs via basis layers , ŷ ` = g f ` ( θ f ` ) and x̂ ` = g b ` ( θ b ` ) . Its operation is described by the following equations : ŷ ` = dim ( θ f ` ) ∑ i=1 θ f ` , iv f i , x̂ ` = dim ( θ b ` ) ∑ i=1 θ b ` , iv b i . Here v fi and v b i are forecast and backcast basis vectors , θ f ` , i is the i-th element of θ f ` . The function of gb ` and g f ` is to provide sufficiently rich sets { v f i } dim ( θ f ` ) i=1 and { vbi } dim ( θ b ` ) i=1 such that their respective outputs can be represented adequately via varying expansion coefficients θ f ` and θ b ` . As shown below , gb ` and g f ` can either be chosen to be learnable or can be set to specific functional forms to reflect certain problem-specific inductive biases in order to appropriately constrain the structure of outputs . Concrete examples of gb ` and g f ` are discussed in Section 3.3 .

Summary: The paper investigates a pure deep learning architecture for Univariate time series analysis by simply ensembling feed-forward networks, along with the residual stacking mechanism for fluid learning. Each of the generic block consists of 4 FC layers followed by the use of forward and backward predictor to have forecast and backcast output of the original input. These blocks forms the stacks, where each stack provides the residuals and the forecast responses further to the next stacks, which ultimately provide the global forecast. To make the internal stack outputs interpretable, assumptions are imposed on the trend model, which follows a polynomial function of time vector, and seasonality model which follows periodic Fourier series. Further, the ensembling of models based on different metrics and input windows is used for better accuracy.