## 2 Fundamental of Phase Space Reconstruction and Deep Neural Network
### Deep Neural Network  
A neural network generally consists of three layers: An input layer, a hidden layer, and an output layer. As shown in Figure 1, it is a simple neural network with three inputs and two outputs and four neurons in a single hidden layer. The neurons between layers are connected by weight \(\omega\). The DNN contains multiple hidden layers, which has a significant improvement in the nonlinear fitting ability of the DNN compared with the single hidden layer. However, too many hidden layers are likely to cause over-fitting. The learning process of the network is the process of adjusting and determining the connection weights \(\omega\) of each neuron through training samples. In this paper, the LMBP algorithm is used to train DNN. Compared with the traditional back propagation (BP) algorithm, the LMBP algorithm has faster convergence speed and higher convergence reliability. It is more suitable for training the DNN and can also satisfy the requirements of real-time ultrashort-term prediction. Unlike the traditional BP algorithm, which uses a gradient descent, the LMBP algorithm is based on the Gauss-Newton method of the least squares solution and takes the square of error \(\mathbf{v}\) as the objective function. \[E(\mathbf{\omega})\ =\ \mathbf{v}^{\rm T}(\mathbf{\omega})\mathbf{v}(\mathbf{\omega}). \tag{7}\]  
The second-order Taylor expansion and derivation of the objective function of Equations (2)-(7) can obtain the change of weight \(\mathbf{\omega}\) as follows:  
\[\Delta\mathbf{\omega}\ =\ -\Big{[}\
abla^{2}E(\mathbf{\omega})\Big{]}^{-1}\
abla E(\mathbf{ \omega}). \tag{8}\]  
where:  
\[\left\{ \begin{array}{l}\
abla E(\mathbf{\omega})\ =\ 2\mathbf{J}^{\rm T}(\mathbf{\omega}) \mathbf{v}(\mathbf{\omega})\\ \
abla^{2}E(\mathbf{\omega})\ =\ 2\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{J}(\mathbf{\omega})+2 \
abla^{2}\mathbf{v}^{\rm T}(\mathbf{\omega})\mathbf{v}(\mathbf{\omega})\end{array} \right.. \tag{9}\]  
\(\mathbf{J}(\mathbf{\omega})\) is the Jacobian matrix of \(\mathbf{v}(\mathbf{\omega})\). If \(\mathbf{v}(\mathbf{\omega})\) consists of \(a\) elements, \(\mathbf{J}(\mathbf{\omega})\) can be written as follows:  
\[\mathbf{J}(\mathbf{\omega})\ =\ \left[ \begin{array}{cccc}\frac{\partial v_{1}}{ \partial\omega_{1}}&\frac{\partial v_{1}}{\partial\omega_{2}}&\cdots&\frac{ \partial v_{1}}{\partial\omega_{a}}\\ \frac{\partial v_{2}}{\partial\omega_{1}}&\frac{\partial v_{2}}{\partial \omega_{2}}&\cdots&\frac{\partial v_{2}}{\partial\omega_{a}}\\ \vdots&\vdots&\ddots&\vdots\\ \frac{\partial v_{a}}{\partial\omega_{1}}&\frac{\partial v_{a}}{\partial \omega_{2}}&\cdots&\frac{\partial v_{a}}{\partial\omega_{a}}\end{array} \right]. \tag{10}\]  
Since \(2\
abla^{2}\mathbf{v}^{\rm T}(\mathbf{\omega})\mathbf{v}(\mathbf{\omega})\) is usually negligible, Formula (8) can be rewritten as follows:  
\[\Delta\mathbf{\omega}\ =\ -\Big{[}\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{J}(\mathbf{\omega}) \Big{]}^{-1}\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{v}(\mathbf{\omega}). \tag{11}\]  
Considering that \(\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{J}(\mathbf{\omega})\) may be irreversible, Formula (11) is modified by adding the correction coefficient \(\mu\), where \(\mathbf{I}\) is the unit matrix. \[\Delta\mathbf{\omega}\ =\ -\Big{[}\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{J}(\mathbf{\omega})+ \mu\mathbf{I}\Big{]}^{-1}\mathbf{J}^{\rm T}(\mathbf{\omega})\mathbf{v}(\mathbf{\omega}). \tag{12}\]  
Figure 1: Single-hidden-layer neural network. Similar to the BP algorithm, the modification of weight \(\mathbf{\omega}^{(k+1)}\) in the \(k\)th iteration is shown in Formula (13). \(\Delta\mathbf{\omega}^{(k)}\) can be obtained by Formula (12). When \(E\Big{(}\mathbf{\omega}^{(k+1)}\Big{)}<\varepsilon\), the algorithm has converged, where \(\varepsilon\) is the given error limit. \[\mathbf{\omega}^{(k+1)}\ =\ \mathbf{\omega}^{(k)}+\Delta\mathbf{\omega}^{(k)}. \tag{13}\]  
The initial value of \(\mu\) generally takes a small positive number such as 0.001.