## 2 Fundamental of Phase Space Reconstruction and Deep Neural Network
### Determination of the Structure of Deep Neural Networks  
The determination of the structure of the DNN is a link of the neural network hyper-parameter adjustment. An unreasonable structure can make the prediction results of the DNN seriously deviate. If the training time is too long, the work is half the effort. The specific method of determination is as follows:  
* Input layer If the DNN is directly trained using the original load data, the determination of the number of neurons in the input layer can be very difficult and requires a lot of debugging to obtain the optimal value. Moreover, when the training set data changes, previous optimal values may no longer be applicable, and the structure of the input layer must be re-debugged. In this model, the input data of the DNN is the phase space reconstructed matrix. Therefore, the number of neurons in the input layer is directly determined by the embedding dimension \(m\) obtained by the C-C method without artificial designation or after debugging to select the optimal value.
* Hidden layer The number of hidden layers can be heuristically determined. When there are few hidden layers, the model will have an under-fitting and cause a large deviation in the predicted value.  
Figure 2: Forecasting model flow chart.  
The prediction model in this paper adopts the one-step prediction. Only the load value at the next moment is predicted at a given time. Therefore, after the load time series is projected to the moving point in the phase space, the model must output the position vector of the point in the phase space of the next time. In fact, if the input of the model is \(\mathbf{p}_{i}(1\leq i\leq M)\) in the phase space reconstruction matrix of Formula (15), only \(p_{i+1+(m-1)t}\) is unknown in position vector \(\mathbf{p}_{i+1}\) at the next moment. Therefore, the output layer must only output the predicted value of load \(\hat{p}_{i+1+(m-1)t}\). If \(i+1>M\), the phase space reconstruction matrix of Formula (15) must be extended downward, and \(\mathbf{p}_{i+1}\) is added as a new line. The expression of \(\mathbf{p}_{i+1}\) is shown in Formula (16), where \(p_{i+1+(m-1)t}\) is the true value of the newly measured load.  
\[\mathbf{p}_{i+1}\ =\ \left[\begin{array}{cccc}p_{i+1}&p_{i+1+t}&\cdots&p_{i+1+(m- 1)t}\end{array}\right] \tag{16}\]  
\(\mathbf{p}_{i+1}\) is used as the input of the model to obtain the predicted value of \(p_{i+2+(m-1)t}\); then, the matrix is augmented, and the predicted value of \(p_{i+3+(m-1)t}\) is obtained, and the process continues until the end of the prediction. The activation function of the output layer is a linear function. The DNN structure is shown in Figure 3.  
DNN can be widely used in different areas. In addition to load forecasting, DNN can also solve the problems including image processing, speech recognition, and fault diagnosis. The training method and network structure are basically the same. The difference is the training data. For the prediction of time series load data, the output layer is the actual load data. For the image processing and fault diagnosis, the output layer is the data label. The structure of DNN is basically the same as the traditional ANN, but the training method is improved which make it have better performance.  
Figure 3: Deep neural network structure.