# Variable Selection in Time Series Forecasting Using Random Forests
## 2 Methods and Data
### Methods  
#### 2.1.1 Definition of ARMA and ARFIMA Models  
Here, we provide the definitions of the autoregressive moving average (ARMA) and ARFIMA stochastic processes, while the reader is also referred to [45] (pp. 6-65, 489-494) for further information. A time series in discrete time is defined as a sequence of observations \(x_{1}\), \(x_{2}\),..., of a certain phenomenon, while the time \(t\) is stated as a subscript to each value \(x_{t}\). A time series can be modelled by a stochastic process. The latter is a sequence of random variables \(\underline{x}_{1}\), \(\underline{x}_{2}\),... Random variables are underlined according to the notation used in [53].  
Let us consider a stationary stochastic process of normally-distributed random variables. The mean \(\mu\) of the stochastic process is defined by:  
\[\mu:=\mathbb{E}[\underline{x}_{t}], \tag{1}\]  
The standard deviation function \(\sigma\) of the stochastic process is defined by:  
\[\sigma:=(\text{Var}[\underline{x}_{t}])^{0.5} \tag{2}\]  
The covariance function between \(\underline{x}_{t}\) and \(\underline{x}_{t+k}\), \(\gamma_{k}\) of the stochastic process is defined by:  
\[\gamma_{k}:=\mathbb{E}[(\underline{x}_{t}-\mu)(\underline{x}_{t+k}-\mu)] \tag{3}\]  
The correlation function between \(\underline{x}_{t}\) and \(\underline{x}_{t+k}\), \(\rho_{k}\) of the stochastic process is defined by:  
\[\rho_{k}:=\gamma_{k}/\sigma^{2} \tag{4}\]  
A normal stationary stochastic process \(\{\underline{a}_{t}\}\) is called a white noise process, if it is a sequence of uncorrelated random variables. Let us consider hereinafter that the white noise is a variable with zero mean, unless mentioned otherwise, and standard deviation \(\sigma_{a}\).  
We define the stochastic process \(\{\underline{y}_{t}\}\) by:  
\[y_{t}:=\underline{x}_{t}-\mu \tag{5}\]Let us consider the operator B, which is defined by:  
\[\text{Bj}\underline{x}_{t}=\underline{x}_{t-j} \tag{6}\]  
Then the operator \(\phi_{p}(\text{B})\) is defined by:  
\[\phi_{p}(\text{B}):=(1-\phi_{1}\text{B}-\ldots-\phi_{p}\text{B}^{p}) \tag{7}\]  
The stochastic process \(\{\underline{x}_{t}\}\) is an autoregressive AR(\(p\)) model, if:  
\[\phi_{p}(\text{B})\underline{y}_{t}=\underline{a}, \tag{8}\]  
which can be written in the following form:  
\[\underline{y}_{t}=\phi_{1}\underline{y}_{t-1}+\ldots+\phi_{p}\underline{y}_{ t-p}+\underline{a}_{t} \tag{9}\]  
Let us also consider the operator \(\theta_{q}(\text{B})\), which is defined by:  
\[\theta_{q}(\text{B}):=(1+\theta_{1}\text{B}+\ldots+\theta_{q}\text{B}^{q}) \tag{10}\]  
The stochastic process \(\{\underline{x}_{t}\}\) is a moving average MA(\(q\)) model, if:  
\[\underline{y}_{t}=\theta_{q}(\text{B})\,\underline{a}_{t}, \tag{11}\]  
which can be written in the following form:  
\[\underline{y}_{t}=\underline{a}_{t}+\theta_{1}\underline{a}_{t-1}+\ldots+ \theta_{q}\underline{a}_{t-q} \tag{12}\]  
The stochastic process \(\{\underline{x}_{t}\}\) is an autoregressive moving average ARMA(\(p\), \(q\)) model, if  
\[\phi_{p}(\text{B})\underline{y}_{t}=\theta_{q}(\text{B})\underline{a}_{t}, \tag{13}\]  
which can be written in the following form:  
\[\underline{y}_{t}=\phi_{1}\underline{y}_{t-1}+\ldots+\phi_{p}\underline{y}_{ t-p}+\underline{a}_{t}+\theta_{1}\underline{a}_{t-1}+\ldots+\theta_{q} \underline{a}_{t-q} \tag{14}\]  
Let \(d\in(-0.5\), \(0.5)\). The stochastic process \(\{\underline{x}_{t}\}\) is an ARFIMA(\(p\), \(d\), \(q\)), if  
\[\phi_{p}(\text{B})(1-\text{B})^{d}\underline{x}_{t}=\theta_{q}(\text{B}) \underline{a}_{t} \tag{15}\]