# Variable Selection in Time Series Forecasting Using Random Forests
## 2 Methods and Data
### Methods
#### 2.1.9 Metrics  
Here we define the metrics used in the comparisons. The error (E) is defined by:  
\[\mathrm{E}:=\psi-x_{n+1} \tag{21}\]  
The absolute error (AE) is defined by:  
\[\mathrm{AE}:=\mid\psi-x_{n+1}\mid \tag{22}\]  
The squared error (SE) is defined by:  
\[\mathrm{SE}:=(\psi-x_{n+1})^{2} \tag{23}\]  
The percentage error (PE) is defined by:  
\[\mathrm{PE}:=(\psi-x_{n+1})/x_{n+1} \tag{24}\]  
The absolute percentage error (APE) is defined by:  
\[\mathrm{APE}:=\mid\psi-x_{n+1}/x_{n+1}\mid \tag{25}\]  
The metrics defined in Equations (21)-(25) can be used for the assessment of the performance in single experiments. To assess the forecasting performance using a set of multiple simulations, the metrics must be summarized. Let \(N\) be the number of the conducted forecasting experiments. Let the serial number of each experiment \(i\) be stated as a subscript to each metric value \(\mathrm{E}_{i}\), \(\mathrm{AE}_{i}\), \(\mathrm{SE}_{i}\), \(\mathrm{PE}_{i}\) and \(\mathrm{APE}_{i}\). Then, the mean of the errors (MoE) is defined by:  
\[\mathrm{MoE}:=(\mathrm{E}_{1}+\dots+\mathrm{E}_{N})/N \tag{26}\]  
The median of the errors (MdoE) is defined by:  
\[\mathrm{MdoE}:=\mathrm{median}(\mathrm{E}_{1},\dots,\mathrm{E}_{N}) \tag{27}\]  
The mean of the absolute errors (MoAE) is defined by:  
\[\mathrm{MoAE}:=(\mathrm{AE}_{1}+\dots+\mathrm{AE}_{N})/N \tag{28}\]  
The median of the absolute errors (MdoAE) is defined by:  
\[\mathrm{MdoAE}:=\mathrm{median}(\mathrm{AE}_{1},\dots,\mathrm{AE}_{N}) \tag{29}\]  
The mean of the squared errors (MoSE) is defined by:  
\[\mathrm{MoSE}:=(\mathrm{SE}_{1}+\dots+\mathrm{SE}_{N})/N \tag{30}\]  
The median of the squared errors (MdoSE) is defined by:  
\[\mathrm{MdoSE}:=\mathrm{median}(\mathrm{SE}_{1},\dots,\mathrm{SE}_{N}) \tag{31}\]  
The mean of the percentage errors (MoPE) is defined by:  
\[\mathrm{MoPE}:=(\mathrm{PE}_{1}+\dots+\mathrm{PE}_{N})/N \tag{32}\]The median of the percentage errors (MdoPE) is defined by:  
\[\text{MdoPE}:=\text{median}(\text{PE}_{1},\,\ldots\,,\,\text{PE}_{N}) \tag{33}\]  
The mean of the absolute percentage errors (MoAPE) is defined by:  
\[\text{MoAPE}:=(\text{APE}_{1}+\ldots\,+\text{APE}_{N})/N \tag{34}\]  
The median of the absolute percentage errors (MdoAPE) is defined by:  
\[\text{MdoAPE}:=\text{median}(\text{APE}_{1},\,\ldots\,,\,\text{APE}_{N}) \tag{35}\]  
The means and the medians can substantially differ when outliers are observed; therefore, they are both useful in the assessment of the forecasting performance. Another useful metric of the forecasting performance, when assessing multiple experiments simultaneously, is the slope of the regression. Let \(i\) also be stated as a subscript to each forecast \(\psi_{i}\) and its corresponding true value \(x_{n+1,i}\). The regression coefficient \(a\) (or slope of the regression) is estimated to measure the dependence of \(\psi_{1}\),..., \(\psi_{N}\) on \(x_{n+1,1}\),..., \(x_{n+1,N}\), when this dependence is expressed by the following linear regression model:  
\[\psi_{i}=a\ x_{n+1,i}+b \tag{36}\]  
In Table 4, we present the range of the values of the metrics and their values when the forecast is perfect.  
A discussion on the appropriateness of the metrics to measure the performance of the forecasting methods can be found in [57] (Chapter 2.5) and [63], while [64; 65] extensively discuss metrics connected to the regression coefficient. Here we preferred to use the metrics of Table 4, due to their simplicity.