## 3 DSPOT-Enhanced Self-Evolving Neural Networks
### Self-Evolving Neural Network  
A self-evolving neural network (SEN) will be built for each dataset, \(X^{class}_{up}\), \(X^{class}_{down}\), and \(X^{class}_{non}\). When training the neural networks (NNs), each element \(P_{class}(t+1)\) in \(X^{class}_{i}\) is treated as the label and the corresponding features contain the wind speed \(W_{s}(t)\), the change in wind direction degree \(W_{dir}(t)\), and current power measurements \(\{P_{class}(t),P_{class}(t-1),\ldots,P_{class}(t-Lag)\}\), where \(Lag\) depends on the measurements (see the discussion in Section 4.1). As demonstrated in Figure 5, NEAT [39] is used to train an NN. NEAT leverages a genetic algorithm (GA) to evolve the NN. It obtains the best network topology and the best weighting parameters by minimizing the forecast error, i.e., \(\min\sum_{l}(\hat{P}_{class}(t)-P_{class}(t))^{2}\), where \(\hat{P}_{class}(t)\) denotes the forecast from the NN.  
As demonstrated in Figure 5, the workflow of NEAT contains random population generation, crossover, mutation, speciation, and evaluation by the fitness function. In this paper, the fitness function is defined using the forecast accuracy:  
\[Fit=-\sum_{l}(\hat{P}_{class}(t)-P_{class}(t))^{2}. \tag{13}\]Each gene in the population set corresponds to a neural network. We aim to find the best gene with the largest fitness value (i.e., the lowest prediction error). In NEAT, the topology of an NN is directly encoded into the gene by a direct encoding scheme [45] in order to avoid Permutations Problem [46] and Competing Conventions Problem [47]. Specifically, connection and node (list of inputs, hidden nodes, and outputs) are encoded. Every unit of connection gene describes the connection weight (W), output node (O), input node (I), enable gate (E), and the number of innovation (N) that corresponds to a consecutive arrangement of new generated node. The workflow of NEAT will be elaborated in the following.  
First, initial population (i.e., a set of genes) is generated randomly. Each gene represents an NN. Note that under this random generation, a neural network might contain no route from inputs to outputs, and we will remove these NNs from the initial population. For example, Figure 6 shows an NN containing 3 inputs (\(P_{class}(t),W_{s}(t),W_{dir}(t)\)) and 1 output (\(\hat{P}_{class}(t+1)\)), where in the first unit of connect gene, I:1 O:5 W:0.5 indicates connection from Node 1 to Node 5 with weight of 0.5, and E:1 means that this is an enabled connection.  
After generating the initial population, NEAT iteratively optimizes the topology and connection weights of NNs using crossover and mutation. Specifically, nodes and connections of NNs are inserted or removed randomly based on the Poisson distribution [39]. For example, Figures 7 and 8 show possible mutations by appending a connection and a node to a neural network, respectively. After crossover and mutation, topologically homogeneous genes are classified as one speciation determined by compatibility distance [39].  
Then, the fitness of species will be evaluated. If the highest fitness of species does not increase or the number of generations is achieved, NEAT will output the species with high fitness value, which will be used for wind generation forecast.  
Figure 5: Workflow of NEAT.  
Figure 6: Encoding of an NN with 1 output and 3 inputs.