# Wind Power Forecasting with Deep Learning Networks: Time-Series Forecasting +
## 3 Wind Power Forecasting Model with Temporal Convolutional Networks
### Parameter Selection for TCN Model Using Evolutionary Algorithm
#### 3.2.2 Analysis of Architecture Design  
Following the architecture analysis step of detailed model design, experiments were conducted with model training in order to validate the optimal parameters of the developed TCN model based on training samples. In the experiment, differential evolution (DE) methods [24; 25; 26; 27; 28] explored these solutions to handle the hyper-parameter tuning of the TCN model for predicting wind power output in order to reach the satisfactory prediction accuracy for different weather conditions. Essentially, the DE method is a population-based stochastic search process using the distance and direction information from the current population to conduct its search. Inspired by [25], we selected the DE method to solve our problem because the historical data for wind power generation are generally not continuous, noisy, or changeable over time; thus, the gradient of the problem being optimized is not used. For all experiments, 50 independent runs were conducted for each test function. The parameter settings for the DE algorithm are listed in Table 5. As shown in Table 5, the following parameters were chosen for the application of DE: population size \(NP=10\); scaling_rate \(F=0.5\), crossover_rate \(CR=0.3\), generation number \(G=30\), and maximum iteration \(=500\). Optimization was terminated at the pre-specified number of generations. Then, nine trial vectors \(V_{i}^{G}\) and difference vectors (Table 6) were generated for each generation \(G\) according to the mutation, crossover and selection operations using Equations (1)-(9). If the trail vector yielded a lower error value of objective function than a predetermined population member, the newly generated trail vector replaced the target vector and was compared in the following generation. \begin{table}
\begin{tabular}{c c c c} \hline  & **Maximum** & **Minimum** & **Average** \\ \hline Training accuracy & 98.2\% & 97.1\% & 97.8\% \\ \hline Test accuracy & 96.7\% & 92.9\% & 95.1\% \\ \hline Average & 97.45\% & 95.0\% & 96.4\% \\ \hline \end{tabular}
\end{table} 
Table 4: Accuracy analysis of the TCN model architecture. \begin{table}
\begin{tabular}{c c c c c} \hline  & **Number of** & **Kernel\_Size** & **Dilatations** & **Number of** & **Optimizer** \\ \hline Parameter & 32 & 10 & [1; 2; 4; 8; 16] & 2 & Adam \\ Value & & & & \\ \hline \end{tabular}
\end{table} 
Table 3: Architecture of the proposed TCN model. For analysis of the optimal parameters of the TCN model, DE allows for the process of mutation, crossover, and selection until the termination condition is satisfied, using Equations (1)-(9); the analysis process is shown in Figure 8.  
In our experiment, the error value and error reduction speed of the loss function were adopted to evaluate whether the appropriate model parameters had been selected. If the convergence error decrease was not smooth (i.e., a bouncing phenomenon), the cost function or model parameters required adjustment. The convergence error of cost function in the training process for TCN model is illustrated in Figure 9. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Target Vector** & **Filter\_Size** & **Dilatation** & \
\begin{tabular}{c} **Number of** \\ **Stacks** \\ \end{tabular} &
\begin{tabular}{c} **Activation** \\ **Function** \\ \end{tabular} & **Optimizer** \\ \hline \#1 & 8 & \([1,2,4,8]\) & 2 & norm\_relu & RMSporp \\ \hline \#2 & 32 & \([1,2,4,8,16]\) & 2 & wavenet & RMSporp \\ \hline \#3 & 16 & \([1,2,4,8]\) & 4 & norm\_relu & Adam \\ \hline \#4 & 8 & \([1,2,4,8]\) & 3 & norm\_relu & SGD \\ \hline \#5 & 32 & \([1,2,4,8]\) & 3 & wavenet & RMSporp \\ \hline \#6 & 8 & \([1,2,4,8]\) & 2 & wavenet & SGD \\ \hline \#7 & 16 & \([1,2,4,8,16]\) & 3 & norm\_relu & SGD \\ \hline \#8 & 16 & \([1,2,4,8]\) & 2 & wavenet & SGD \\ \hline \#9 & 32 & \([1,2,4]\) & 4 & norm\_relu & Adam \\ \hline \hline \end{tabular}
\end{table} 
Table 6: The parameter settings for the nine trial vectors. Figure 8: Differential evolution used to decide the optimal parameters of TCN model training.