# Robust Sales forecasting Using Deep Learning with Static and Dynamic Covariates
## 2 Autoregressive Neural Network Model  
In this research, we employ a deep learning RNN sequence-to-sequence model known as DeepAR [9], which represents a state-of-the-art algorithm specifically designed to tackle the intricate challenges inherent in time series forecasting. Developed by Amazon Web Services (AWS), DeepAR has demonstrated excellent performance across diverse domains such as finance, healthcare, and supply chain management. This success can be attributed to its inherent capability to capture complex temporal patterns and dependencies within data, which may remain obscured when using traditional forecasting methods [10; 11]. A notable advantage of DeepAR lies in its built-in ability to model uncertainty, offering probabilistic forecasts [12; 13]. This feature proves particularly crucial in decision-making processes that demand a nuanced assessment of both risks and opportunities. Furthermore, DeepAR's flexibility in accommodating varying data characteristics and its adaptability to incorporate external covariates make it a versatile tool for enhancing the accuracy and reliability of predictions. Let \(z_{i,t}\) denote the sales of product \(i\) at time \(t\). The primary objective of DeepAR model is to forecast the conditional probability \(P\) of future sales \(z_{i,t_{0}:T}\) using past sales \(z_{i,t_{0}-1}\) and additional information in the form of covariates \(\mathbf{x}_{i,1:T}\), where \(t_{0}\) represents the first time instant of the future and \(T\) represents the last time instant of the future [14]:  
\[P(z_{i,t_{0}:T}|z_{i,1:t_{0}-1},\mathbf{x}_{i,1:T}). \tag{1}\]  
It is important to note that the time index \(t\) is relative, meaning that \(t=1\) may not correspond to the initial time point of the time series. During training, we have access to \(z_{i,t}\) in both the conditioning range \([1,t_{0}-1]\) and the prediction range \([t_{0},T]\). The former is used for encoding, while the latter is used for decoding in the sequence-to-sequence model. However, during inference (when we make predictions), \(z_{i,t}\) is not available in the prediction range. At each time step \(t\), the model produces an output represented by \(\mathbf{h}_{i,t}\):  
\[\mathbf{h}_{i,t}=h(\mathbf{h}_{i,t-1},\mathbf{z}_{i,t-1},\mathbf{x}_{i,t}; \Theta). \tag{2}\]  
This output is obtained by applying a multi-layer RNN with long short-term memory (LSTM) cells [15] parameterized by \(\Theta\). The model is considered autoregressive because it takes the sales value from the previous time step \(z_{i,t-1}\) as input. Additionally, it is considered recurrent, as the output from the network at the previous time step \(\mathbf{h}_{i,t-1}\) is given back as input at the next time step. During training, we learn the model parameters by maximizing the log-likelihood of a chosen probability distribution using the equation:  
\[L=\sum_{i=1}^{N}\sum_{t=0}^{T}\log L(z_{i,t}|\theta(\mathbf{h}_{i,t})). \tag{3}\]  
Here, \(L\) denotes the likelihood of the distribution, \(N\) corresponds to the number of products, and \(\theta\) represents a linear mapping from the function \(\mathbf{h}_{i,t}\) to the parameters of the distribution. DeepAR uses the entire time range to calculate the loss since the encoder model is identical to the decoder. DeepAR forecasts a single value at each step. However, during the inference phase, to predict several steps, the model repeatedly obtains forecasts for subsequent periods until the forecast horizon is reached. The model starts by generating samples from a probability distribution that has been trained on past sequences. The first prediction is made using these samples, and this prediction is then used as input to the model to make the next prediction. This process is repeated for each subsequent period. Because the predictions are derived from samples taken from the trained distribution, the model's output is probabilistic. This means that the model does not produce a single deterministic value for each prediction, but rather a distribution of possible values.