# Wind Power Forecasting with Deep Learning Networks: Time-Series Forecasting +
## 3 Wind Power Forecasting Model with Temporal Convolutional Networks
### Parameter Selection for TCN Model Using Evolutionary Algorithm
#### 3.2.1 Preliminary Architecture Design  
In this step, four crucial architecture parameters were selected by transferring learning cases in the TCN predictor [20; 21; 22; 23]; the original parameters of TCN models developed for wind power prediction models were obtained from [PERSON] at GitHub [29], namely, (i) number of filters, (ii) activation function, (iii) optimizer, and iv) dilatation coefficient, in order to decide the initial model architecture for model training.  
In practice, the cost function is a measure of the inaccuracy of the model in terms of the difference between predicted values and real measured values. Following the analysis of three filter sizes (8, 16, and 32), as described in Figure 4, the filter size of 32 exhibited the smallest convergence error of the cost function after 100 iterations of simulation and was the optimal choice for the designed TCN-based prediction model.  
In artificial neural networks, the activation function of a node defines the output of that node based on the input or set of inputs. Generally, nonlinear activation functions allow such networks to compute complex problems using a few nodes. In the experiment, we analyzed two types of nonlinear activation function, the norm_relu and Tanh*Sigmoid activation function used in WaveNet. The corresponding convergence error of the cost function for these activation functions is presented in Figure 5; norm_relu was selected as the activation function of the model.  
Figure 4: Convergence error of cost function with three different filter sizes.  
Figure 5: Convergence error of cost function for two nonlinear activation functions.  
3. Optimizer To minimize loss during model training, an optimizer was adopted to improve the accuracy of the model through adjustment of the filter weights. We assessed three popular optimizers in the TCN model, Adam, SGD, and RMSprop; the corresponding convergence error of cost function is detailed in Figure 6. The Adam optimizer was chosen as the ideal optimizer for model training.
4. Dilatation coefficient Following the analysis of three combinations of dilatation coefficients (1, 2, 4; 1, 2, 4, 8; 1, 2, 4, 8, 16) in the model input architecture, the third set (1, 2, 4, 8,16) achieved the most optimal results in relation to the error convergence of the cost function, as depicted in Figure 7.  
The architectural components of the proposed TCN model for transferring learning from model training are listed in Table 3.  
Figure 6: Convergence error of cost function for three types of optimizer.  
Figure 7: Convergence error of cost function with three sets of dilatation coefficients.  
To confirm the appropriate architecture of the model design, different model architecture parameters were experimentally investigated. In this experiment, 83.3% (500,000 records) of the data from the open dataset of wind farm in Turkey (2018) [30] was randomly selected to serve as the training dataset, and the remaining 17.7% (100,000 records) was used for testing. The average accuracy of the results of experimental training and testing is summarized in Table 4. Model accuracy was above 96.4% based on different parameter combinations, with a low convergence loss result for the cost function. Thus, the architectural parameters of Table 3 for the TCN model were validated.