## 3 Preliminaries
### Time Series Forecasting
#### 3.2.1 Performance Metrics  
We considered the following error and accuracy metrics to evaluate our forecasting model. The equation of the performance metrics are summarized in Table 1.  
* **Mean Absolute Percentage Error (MAPE):** The Mean Squared Error (MSE) is the average of the square of the forecast error. As the square of the errors are taken, the effect is that larger errors have more weight on the score. In this case, MAPE comes in handy where the data are not understood from the error measure itself. As MAPE is a percentage error, it gives a good idea of the relative error. MAPE is the most widely used unit-free measure.
* **Direction accuracy:** DAC is a measure of prediction accuracy of a forecasting method in statistics. It compares the forecast direction (upward or downward) to the actual realized direction. DAC is similar to a binary evaluation. The metric only considers the upward or downward direction in the time series and is independent of the quantitive value of the increase or decrease. DAC provides the probability that the forecasting method can detect the correct direction of the time series [41].
* **Relative Absolute Error (RAE):** Relative measures give an indication of how well the forecaster's predictions are doing compared to just using the last known target value as the prediction. For RAE, the comparison is done with respect to absolute error with  
\begin{table}
\begin{tabular}{l l l} \hline \hline
**Metric** & **Equation** & **Notation** \\ \hline Mean Absolute Percentage Error (MAPE) & \(MAPE=\frac{100\%}{N}\sum_{i=1}^{N}\frac{|\mathcal{D}-X_{i}|}{\mathcal{X}_{i}}\) & \(N=\) Sample Size \\ Direction Accuracy (DAC) & \(DAC=\frac{1}{N}[count(sign(X_{cut}-X_{pr})==\) & \(P_{i}=\) Predicted Value \\  & \(sign(P_{cut}-P_{pr}))]\) & \(X_{cut}=\) Actual Value \\ Relative Absolute Error (RAE) & \(RAE=\frac{\sum_{i=1}^{N}|P_{i}-X_{i}|}{\sum_{i=1}^{N}|P_{i}-X_{i}|}\) & \(X_{pr}=\) Actual Current \\ Root Relative Squared Error (RRSE) & \(RRSE=\frac{\sqrt{\sum_{i=1}^{N}\frac{(\mathcal{D}-X_{i})^{2}}{N}}}{\sqrt{\sum_{ i=1}^{N}\frac{(\mathcal{D}-X_{i})^{2}}{N}}}\) & \(P_{cut}=\) Predicted Current \\ Root Mean Squared Error (RMSE) & \(RMSE=\sqrt{\sum_{i=1}^{N}\frac{(\mathcal{D}-X_{i})^{2}}{N}}\) & \(P_{pr}=\) Predicted Previous \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance evaluation metrics for time series forecasting.  
the last known target. RAE takes the total absolute error and normalizes it by dividing by the total absolute error of the simple predictor (i.e., the last known target).
* **Root Relative Squared Error (RMSE):** RMSE is similar to RAE. The difference is that RRSE takes the square root of the total squared error and normalizes it by dividing by the square root of the total squared error of the simple predictor (i.e., the last known target).
* **Root Mean Squared Error (RMSE):** MSE is more vulnerable to outliers since it gives extra weight to large errors. In addition, the squared errors are on different scale from the original data. Thus, RMSE, which is the square root of MSE, is often preferred to MSE as it is on the same scale as the data. However, RMSE is also sensitive to forecasting outliers. Researchers now seem to prefer unit-free measures for comparing methods [42; 43].