# Wind Power Forecasting with Deep Learning Networks: Time-Series Forecasting +
## 2 Literature Review
### Differential Evolution Algorithm  
In the design of DNNs for processing time-series data, the optimal parameters of the developed model are identified from training data in order to achieve high predictive precision in the model output. In supervised machine learning algorithms, in order to minimize the convergence loss of the model in the learning process the optimal parameters of the model can be investigated using evolutionary algorithms (EAs). Practically, the EA algorithm is an effective and efficient approach for solving global numerical optimization problems, avoiding overfitting, and preventing the gradient descent algorithms from converging prematurely on a local suboptimal solution. EAs constitute a smart approach to solving constrained multiobjective optimization problems. In practice, EAs are a family of nature-inspired algorithms widely used for solving complex optimization problems which can be used for assisting developers in determining the optimal parameters of the training model. The differential evolution (DE) algorithm [24; 25; 26; 27; 28] is a branch of EA that follows the general procedures of EAs. In detail, DE is a metaheuristic method that optimizes a problem by iteratively attempting to provide an improved candidate solution with regard to a set measure of quality. DE was introduced by [PERSON] and [PERSON] in the 1990s [16], and is applied to solve multiobjective optimization with constraints. Typically, metaheuristic methods can search large spaces for candidate solutions. DE is particularly used for multidimensional real-valued functions; however, it does not use the gradient of the problem being optimized and therefore does not require the optimization problem to be differentiable. Thus, DE can be used on optimization problems that are not continuous, noisy, or changeable over time. Figure 1: Basic architecture of a temporal convolutional network for wind power prediction. The three basic operators of the DE algorithm are the mutation, crossover, and selection operators. The fundamental idea behind DE is a scheme for producing trial vectors according to the manipulation of target vector and difference vector. If the trail vector yields a lower objective function than a predetermined population member, the newly generated trail vector will replace the vector and be compared in the following generation [28]. After the initialization process, DE forms a loop of the mutation, crossover, and selection processes until the termination condition is satisfied [24]. The processes of these operators are described as follows:  
(i) Initialization  
Suppose that each individual of the population is denoted as \(X_{i}=[x_{ij}]=(x_{i,1},\ldots,\)\(x_{i,j},\ldots,\)\(x_{i,D})\), where \(i\) = 1, \(\ldots,\)\(N\), \(N\) is the number of the solution as well as \(j\) = 1, \(\ldots,\)\(D\), and \(D\) represents the number of the dimension. \(X_{i}\) is limited by \(X_{min}=(x_{min}^{G},\ldots,\)\(x_{min}^{G},\ldots,\)\(x_{min}^{G},\ldots,\)\(x_{min}^{G})\) and \(X_{max}=(x_{max}^{G},\ldots,\)\(x_{max}^{G},\ldots,\)\(x_{max}^{G})\), which is specified by the user. \(G\) is the generation number. An individual of the population can be defined as follows:  
\[X_{i}=(x_{i,1},\ldots,\)\(x_{i,j},\ldots,\)\(x_{i,D})\), \(i\) = 1, \(\ldots,\)\(N\). \(j\) = 1, \(\ldots,\)\(D\)  
First, the initialization population randomly selects the initial parameter values uniformly based on the intervals \([X_{min},\)\(X_{max}]\). The commonly used initialization method for individuals is  
\[X_{i}^{G}=rand(0,1)\cdot(X_{max}-X_{\min})+X_{min} \tag{2}\]  
where \(rand(0,\) 1) represents the generation of a uniformly distribution random number located in \([0,\)\(1]\). (ii) Mutation  
The \(DE\) algorithm adopts the mutation strategy, in which a mutant vector is created for each individual \(V_{i}^{G}\) (also called the trial vector) in each generation \(G\).