## 3 Popular ANN Architectures Employed for Time Series Forecasting Purposes
### Feedforward Neural Networks
#### 3.1.5 Cross-Validation  
Cross-validation (CV) is a statistical technique for estimating the prediction (or forecasting) accuracy of any model using only available training examples. CV can be helpful when deciding which model to select from a list of properly trained models; we would of course select the model that exhibits the smallest prediction error, as estimated by the CV procedure. CV can also serve as a training framework for ANNs; in fact, CV is often regarded as an integral part of the FFNN model construction process. In general, when training FFNNs for prediction purposes, CV is employed to fine-tune FFNN meta-parameters (such as \(d\), the number of input nodes, and \(q\), the number of hidden units), aiming at reducing over-fitting risk and at the same time improving generalization ability (i.e., prediction accuracy). The idea here is to regard a combination of meta-parameters, say \((d=12,q=3)\), as a unique FFNN model. Following this idea, we would apply CV, for example, to each element in the combination set \(\{(d,q)|d=3,6,12;q=2,3,4,5\}\), generating an estimated CV prediction error for each one of the 12 possible combinations. We would finally keep the combination (i.e., FFNN model) that generates the smallest CV prediction error. So, how does CV work? Typically, we randomly split our set of available training examples into two complementary sets: one set, containing approximately 80% of all training examples, is used exclusively for training the considered model, while the other set, containing the remaining 20% of all training examples, is used exclusively for measuring prediction accuracy by comparing target values and their corresponding model outputs via an error function similar to that employed in BP to quantify training error, e.g., mean squared error. The former of these two complementary sets is obviously called the _training set_ and the latter is called the _validation set_. The 80% and 20% sizes are just a rule of thumb, and other sizes for these two sets could be chosen. So, we use the training set to obtain a fully trained FFNN model via BP, for example, and then feed to this trained FFNN model the input values contained in the examples from the validation set, thus obtaining outputs that are compared against their corresponding target values from the validation set, producing a prediction error measure, \(PEM\). This is the basic CV iteration. We repeat many times the basic CV iteration (random generation computation) in order to generate many \(PEM\) measures, and finally, we average all generated \(PEM\)s. This final \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Example** & \(y_{t-11}\) & \(y_{t-10}\) & \(y_{t}\) & \(y_{t+1}\) \\ \hline
1 & \(y_{1}\) & \(y_{2}\) & \(\ldots\) & \(y_{12}\) & \(y_{13}\) \\
2 & \(y_{2}\) & \(y_{3}\) & \(\ldots\) & \(y_{13}\) & \(y_{14}\) \\
3 & \(y_{3}\) & \(y_{4}\) & \(\ldots\) & \(y_{15}\) & \(y_{16}\) \\ \(\vdots\) & \(\vdots\) & \(\vdots\) & \(\ldots\) & \(\vdots\) & \(\vdots\) \\ \(N-12\) & \(y_{N-12}\) & \(y_{N-11}\) & \(\ldots\) & \(y_{N-1}\) & \(y_{N}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Training dataset for one-step-ahead FFNN time series models. average would be the estimated prediction error produced by the CV procedure. As we can see, this is a procedure that makes intensive use of available computational resources but produces robust results. We can combine CV with regularization for even better results. For more information about CV, early stopping, regularization, and other techniques to improve FFNN performance, see [PERSON] (2016). It is important to keep in mind that to obtain valid results from the CV procedure, we must make sure that our training data examples are independent and come from the same population. Unfortunately, the condition of independence does not hold with time series data, as chronologically ordered observations are almost always serially correlated in time (one exception is white noise).