# Anomaly Detection of Wind Turbines Based on Deep Small-World Neural Network
## 2 Deep Small-World Neural Network (DSWNN)
### Small-World Transformation of DSWNN  
A small-world network is an intermediate network between completely random and completely regular, which was originally proposed by [PERSON] in 1998 to describe the natural distribution of biological, technological and social networks [PERSON] (1998). After that, various researches began to apply the characteristics of a small-world network to the structural improvement of artificial neural networks (ANNs) [PERSON] (1998); [PERSON] (1998); [PERSON] (1998). We summarize the current researches and consider that the small-world transformation has two ways (see Figure 3): reconstruct-edge transformation and add-edge transformation. Taking a four-layer BP neural network with four neurons in each layer as an example, the reconstruct-edge transformation first separates the connections between adjacent-layer neurons randomly, and then reconstructs the new connections between nonadjacent-layer neurons. The add-edge transformation does not have the disconnecting procedure, it only adds new connections between nonadjacent-layer neurons without changing any original edges. The positions of the newly added connections are all randomly distributed among all network neurons, and the degree of network randomization is described by probability \(p\). \[p=n_{ad}/n_{or} \tag{8}\]  
where, \(n_{ad}\) and \(n_{or}\) are the number of the newly added connections and original connections, respectively. Figure 4 gives the random add-edge procedure from a regular network to a random one. When \(p\) = 0 or \(p\) = 1, the network is completely regular or completely random. While when \(p\) is between 0 and 1, the network has a small-world property. To probe the intermediate region 0 \(<p<\) 1, the characteristic path length \(L(p)\) and clustering coefficient \(C(p)\) are used to quantify the small world structural properties. Figure 3: The ways of small-world transformation. The characteristic path length \(L(p)\) is a global property of measuring the average length of all connected edges in a network, and clustering coefficient \(C(p)\) is a local property that is used to describe the density of connected edges in local areas. Figure 5 shows the changing normalized values of \(L(p)\) and \(C(p)\) with \(p\) for the four-hidden-layer DSWNN described in Figure 1. It can be observed that as \(p\) increases, \(L(p)\) drops sharply while \(C(p)\) descends relatively slowly. When \(p\) moves towards 0.1, a large \(C(p)\) and a small \(L(p)\) are obtained, which indicates that the topology of the DSWNN has the best small-world properties. Therefore, the number of newly added edge connections is \(p\) = 0.1 times of the total number of connections. This paper selects the way of add-edge transformation to reconstruct the DSWNN model. Suppose that the DSWNN model has \(H\) (\(H\) = 1, 2, 3,..., \(i\),... ) hidden layers, and there are \(N\) neurons in the \(i\)th hidden layer and \(M\) neurons in the \(i+1\)th hidden layer. When \(p\) = 0, the DSWNN model is a regular network and the weight matrix \(W_{i}\) between the \(i\)th and \(i+1\)th hidden layers is described as Equation (9). Accordingly, the connection matrix \(W\) for the entire hidden layers can be expressed as in Equation (10). \[W_{i}=\left[ \begin{array}{ccccc}w_{i11}&w_{i12}&\ldots&w_{i1M}\\ w_{i21}&w_{i22}&\ldots&w_{i2M}\\ \vdots&\vdots&\ddots&\vdots\\ w_{iN1}&w_{iN2}&\ldots&w_{iNM}\end{array} \right] \tag{9}\]  
\[W=\left[ \begin{array}{ccccc}W_{1}&0&\ldots&0&0&\ldots&0\\ 0&W_{2}&\ldots&0&0&\ldots&0\\ \vdots&\vdots&\ddots&0&0&\ldots&0\\ 0&0&0&W_{i}&0&\ldots&0\\ 0&0&0&0&W_{i+1}&\ldots&0\\ \vdots&\vdots&\vdots&\vdots&\vdots&\ddots&0\\ 0&0&0&0&0&W_{H-1}\end{array} \right]. \tag{10}\]  
Figure 4: The random add-edge procedure from a regular back propagation (BP) neural network to a random one. Figure 5: \(L(p)\) and \(C(p)\) for the four-hidden-layer DSWNN described in Figure 1.  
When \(p=0.1\), the DSWNN does not disconnect connections in the original network and only generates connections between neurons in the non-adjacent hidden layers.