# Estimating Weak Pulse Signal in Chaotic Background with Jordan Neural Network

Authors: Liyun Su; Xiu Ling
Journal: Complexity
 (2020)

---

## Abstract

In target estimating sea clutter or actual mechanical fault diagnosis, useful signal is often submerged in strong chaotic noise, and the targeted signal data are difficult to recover. Traditional schemes, such as Elman neural network (ENN), backpropagation neural network (BPNN), support vector machine (SVM), and multilayer perceptron- (MLP-) based model, are insufficient to extract the weak signal embedded in a chaotic background. To improve the estimating accuracy, a novel estimating method for aiming at extracting problem of weak pulse signal buried in a strong chaotic background is presented. Firstly, the proposed method obtains the vector sequence signal by reconstructing higher-dimensional phase space data matrix according to the Takens theorem. Then, a Jordan neural network- (JNN-) based model is designed, which can minimize the error squared sum by mixing the single-point jump model for targeting signal. Finally, based on short-term predictability of chaotic background, estimation of weak pulse signal from the chaotic background is achieved by a profile least square method for optimizing the proposed model parameters. The data generated by the Lorenz system are used as chaotic background noise for the simulation experiment. The simulation results show that Jordan neural network and profile least square algorithm are effective in estimating weak pulse signal from chaotic background. Compared with the traditional method, (1) the presented method can estimate the weak pulse signal in strong chaotic noise under lower error than ENN-based, BPNN-based, SVM-based, and -ased models and (2) the proposed method can extract the weak pulse signal under a higher output SNR than BPNN-based model.


---

## Body

## 1. Introduction

As early as the end of the 19th century, the French Poincare found that the solution of the three-body problem could be random within a certain range when he studied the three-body problem, but this discovery did not arouse widespread concern among scholars. It was not until 1963 that Lorenz, an American meteorologist, published a paper [1] that gave the initial condition parameters for generating chaos in differential equations, which were called Lorenz equations. With the continuous in-depth study of chaos, the characteristics and scientific concepts of chaos gradually come into people’s vision. Chaotic phenomena can be described as random phenomena in a deterministic system, which is a complex behavior generated by a nonlinear dynamic system. Moreover, chaos has three characteristics: sensitivity to initial value, short-term predictability, and intrinsic randomness.Pulse signal is a kind of discrete signal, which is usually submerged by noise and not easy to detect. It is widely existed in communication engineering, biomedicine, and industrial automatic control. Therefore, the prediction of the pulse signal from chaotic noise has certain practical significance. Experts and scholars at home and abroad have also studied the prediction of the pulse signal in chaotic noise and obtained a lot of research results, such as Volterra filter, correlation detection in time domain, spectral analysis in frequency domain, chaotic Duffing oscillator [2], local linear [3–5], multiple kernel extreme learning machine [6], and Kalman filter , extend Kalman filter (EKF) [7], wavelet transform and nonlinear autoregressive model, etc.At present, experts and scholars have proposed many methods to predict chaotic time series and extract impulse signal from chaotic noise [8, 9]. Jiang Xiangcheng proposed the Volterra adaptive method to predict chaotic time series [10]. The improved local projection algorithm proposed by Han Min is a nonlinear chaotic noise reduction method, which combines wavelet analysis with local projection to estimate noisy chaotic signals [11]. Wang Shiyuan et al. proposed the fractional order maximum correlation entropy algorithm [12].Neural network is a model that imitates the information processing mode of the human nervous system. It is the abstraction and simplification of human neural network. Neural networks can approximate and model the processes of nonlinear dynamic systems [13]. Neural network is divided into forward network, feedback network, random neural network, and competitive network. Because of the rich diversity of neural networks, different neural network to build models of nonlinear systems is also being studied. For example, radial quantity function (RBF) [14, 15], support vector machine (SVM) [16], least squares support vector machine (LS-SVM) [17], backpropagation neural network (BPNN) [13], fuzzy neural network (FNN) [18], recursive neural network (RNN) [19, 20], multilayer perceptron (MLP), nonlinear regression model (NAR), recursive predictor (RPNN), and ESN [21]. Chandra used the method of memetic cooperative coevolution to train the Elman recurrent neural network on the problem of grammatical inference [22]. Two years later, Chandra used Elman neural network to characterize chaotic time series and added cooperation and competition into them, so as to achieve the purpose of prediction [20]. This paper uses the Jordan network, a neural network established by Jordan in 1986 [23]. Both the Jordan neural network and the Elman neural network (ENN) are cyclic neural networks. The difference is that the Elman neural network feeds the output value of the hidden layer at the previous moment back to the feedback layer, while the Jordan neural network feeds the output value of the output layer at the previous moment back to the feedback layer, which can be used to represent the dynamic system.In this paper, the Jordan neural network is used to fit the chaotic background and estimate the pulse signal from its residuals. Because the pulse signal is very weak, the characteristics of chaotic time series are mainly reflected in the chaotic background. Firstly, the observed signal was reconstructed, then the Jordan neural network and the single-point pulse signal model were established, the weight and bias of the Jordan neural network were optimized by the gradient descent algorithm, and then the amplitude of the pulse signal was estimated by the profile least square method.The remainder of the paper is organized as follows. Section2 gives a brief background and related work. Section 3 presents and analyzes this problem. Section 4 estimates the weak pulse signal from a chaotic background by Jordan neural network. Section 5 presents 3 experimental simulations results and discussion. Finally, conclusions and discussion on future work are provided in Section 6.


## 2. Background and Related Work

Our work mainly involves two aspects: the application of feedback neural network and the application of feedback neural network in time series prediction.


### 2.1. Application of Feedback Neural Network

Chaotic time series is a kind of irregular motion in deterministic system and a special time series. However, the traditional time series prediction models—MA, ARMA, and ARIMA—and other models are not ideal for chaotic time series prediction.The applications for chaotic time series prediction are wide and range from financial prediction to weather prediction [20].Recursive neural networks are used in a wide range of applications, such as weather prediction [20], unbalanced fault diagnosis [24], and detection of network malware [25]. Classification is also an important application of recursive neural network. Recently, recursive neural networks have been widely integrated into convolutional neural networks. Epstein integrates the recursive neural network into the convolutional neural network for image classification. Experiments show that the model is more robust to category-independent attributes [26]. Not only is the wide application range of neural network reflected in its high frequency of use in different fields but also it can be flexibly combined with other networks, models, and methods. Slawek Smyl found that the preprocessing parameters were actually updated formulas from some models of the exponential smoothing family. Therefore, it proposes a hybrid prediction method in [27], which mixes the exponential smoothing model with the advanced long and short-term memory neural network in a common framework. The advantages of this hybrid prediction method are that the exponential smoothing equation enables the method to effectively capture the major components of a single sequence, while neural networks allow for nonlinear trending and cross-learning.


### 2.2. Application of Feedback Neural Network in Time-Series Prediction

Liu et al. combined the dual stage two-phase model with the feedback neural network to make a long-term prediction of multiple time series in [28]. And it has been successfully used to develop widely used expert or intelligent systems. In [29], Wei Wu used ARIMA model, Elman neural network, and Jordan neural network, respectively, to predict the nonlinear time series data of human brucellosis in China, and the results showed that the feedback neural network was more accurate than the traditional ARIMA model. In [30], Zhihao Wang proposed a heart sound signal recovery method based on the long and short time memory prediction model based on the structure of circular neuron network. This method can not only restore incomplete or disturbed signals, but also have filtering effect. The damaged or incomplete heart sound signal has a good recovery effect, which has a promoting effect on medical research. Zhang proposed a new nonstationary time series modeling optimization algorithm in [31]. The algorithm uses moving window and exponential decay weight to eliminate the influence of historical gradient. The regretful boundary of the new algorithm is analyzed to ensure the convergence of the calculation. The simulation results of this algorithm on the data set of short-term power load are better than the existing optimization algorithm.


## 2.1. Application of Feedback Neural Network

Chaotic time series is a kind of irregular motion in deterministic system and a special time series. However, the traditional time series prediction models—MA, ARMA, and ARIMA—and other models are not ideal for chaotic time series prediction.The applications for chaotic time series prediction are wide and range from financial prediction to weather prediction [20].Recursive neural networks are used in a wide range of applications, such as weather prediction [20], unbalanced fault diagnosis [24], and detection of network malware [25]. Classification is also an important application of recursive neural network. Recently, recursive neural networks have been widely integrated into convolutional neural networks. Epstein integrates the recursive neural network into the convolutional neural network for image classification. Experiments show that the model is more robust to category-independent attributes [26]. Not only is the wide application range of neural network reflected in its high frequency of use in different fields but also it can be flexibly combined with other networks, models, and methods. Slawek Smyl found that the preprocessing parameters were actually updated formulas from some models of the exponential smoothing family. Therefore, it proposes a hybrid prediction method in [27], which mixes the exponential smoothing model with the advanced long and short-term memory neural network in a common framework. The advantages of this hybrid prediction method are that the exponential smoothing equation enables the method to effectively capture the major components of a single sequence, while neural networks allow for nonlinear trending and cross-learning.


## 2.2. Application of Feedback Neural Network in Time-Series Prediction

Liu et al. combined the dual stage two-phase model with the feedback neural network to make a long-term prediction of multiple time series in [28]. And it has been successfully used to develop widely used expert or intelligent systems. In [29], Wei Wu used ARIMA model, Elman neural network, and Jordan neural network, respectively, to predict the nonlinear time series data of human brucellosis in China, and the results showed that the feedback neural network was more accurate than the traditional ARIMA model. In [30], Zhihao Wang proposed a heart sound signal recovery method based on the long and short time memory prediction model based on the structure of circular neuron network. This method can not only restore incomplete or disturbed signals, but also have filtering effect. The damaged or incomplete heart sound signal has a good recovery effect, which has a promoting effect on medical research. Zhang proposed a new nonstationary time series modeling optimization algorithm in [31]. The algorithm uses moving window and exponential decay weight to eliminate the influence of historical gradient. The regretful boundary of the new algorithm is analyzed to ensure the convergence of the calculation. The simulation results of this algorithm on the data set of short-term power load are better than the existing optimization algorithm.


## 3. Problem Analysis

This paper analyzes the weak pulse signal which is fixed chaotic noise. Lett be a time parameter. We can assume that observation sequence, expressed as xt, contains chaotic noise ct, white noise signal nt, and weak pulse signal st. So we have a formula as follows:(1)xt=st+ct+nt.Pulse signal can be boiled down by periodic signal:(2)st=∑i=1kaisit=aΤst,where(3)a=a1,a2,⋯,akΤ,st=s1t,s2t,⋯,sktΤ,sit=1,t=Ti,2Ti⋯0,others,where aii=1,2,⋯,k is the amplitude of the weak pulse signal siti=1,2,⋯,k and Ti is the periodic of siti=1,2,⋯,k. Estimating weak pulse signal from xt, we can consider that estimating weak pulse signal st in chaotic background is to estimate the value of a. Then, our ultimate goal turns to estimating a.


## 4. Estimating the Pulse Signal from Strong Chaotic Background

### 4.1. Phase Space Reconstruction

Chaotic time series is a univariate or multivariable time series produced by a dynamical system. Phase space reconstruction theorem is mapping a chaotic time series to high-dimensional space. According to the Takens theorem of embedding [32], there is a correlation between one component of the dynamical system and the other components.For an observed chaotic signalxt,t=1,2,⋯,n, the embedding dimension m and the time delay τ could be obtained by using Cao’s method [33], and a phase point in the reconstructed phase space can be expressed as Xt=xt,xt−τ,⋯,xt−m−1τΤ, where t=n1,n1+1,⋯,n and n1=m−1τ+1. Then there is a smooth mapping, g:Rm⟶R, which provides the relationship between Xt and yt, denoted as yt=gXtt=n1,n1+1,⋯,n−1. To predict chaotic time series is to find the function g or the approximate function of g.Remark 1.
In mathematics, Takens embedding theorem gives the conditions under which a chaotic dynamical system can be reconstructed from a sequence of observations of the state of a dynamical system. The reconstruction preserves the properties of the dynamical system that do not change under smooth coordinate changes, but it does not preserve the geometric shape of structures in phase space. It provides the conditions under which a smooth attractor can be reconstructed from the observations made with a generic function.
Cao’s method is a practical method which determines the minimum embedding dimension from a scalar time series. The average ratio of Euclidean distance of a vector with embedded dimensionm to Euclidean distance with embedded dimension m + 1 is denoted as E (m). The ratio of E (m + 1) to E (m) stops changing when m is greater than some value m0 if the time series comes from an attractor. Then m0 + 1 is the minimum embedding dimension we look for.


### 4.2. Jordan Neural Network

Jordan combined the Hopfield network storage concept and distributed parallel processing theory to create a new circular neural network in 1986 [23]. Jordan neural network is made of 4 parts: input layer, hidden layer, output layer, and context layer, respectively. There is a first-order delay operator in the connection from the output layer to the context layer, so that the context layer stores the information of the output layer. There are two types of activation functions in the neural network: linear activation function and nonlinear activation function. Sigmoid nonlinear activation function which is expressed as fx=1/1+e−x is used at the hidden layer and the linear function is used at the output layer in this paper. Jordan neural network topology is shown in Figure 1.Figure 1
Jordan neural network topology.According to Figure1, Xt=xt,xt−τ,xt−2τ,⋯,xt−m−1τΤ is used to represent the input vector of the Jordan neural network, where t=n1,n1+1,⋯,n and n1=1+m−1τ. ht=h1t,h2t,⋯,hrtΤ is the output vector of hidden layer. Wij1i=1,2,⋯,r,j=1,2,⋯,m are the weights from input layer neuron (j) to ith first hidden layer neuron. Wi2i=1,2,⋯,r are the weights from context layer neuron to ith first hidden layer neuron. Wi3i=1,2,⋯,r are the weights from hidden layer neuron (i) to output layer neuron. bi1i=1,2,⋯,r are the biases of hidden layer neuron (i). b2 and b3 are the biases of output layer and context layer, respectively. f2 and f3 are the activation function of output layer and context layer, respectively, and they are usually linear function, while fi1i=1,2,⋯,r are the activation function of hidden layer neuron (i). hiti=1,2,⋯,r is the value of hidden layer neuron (i). dt and yt are the value of context layer and output layer.In order to express clearly and write easily, the following signs will be introduced:(4)W1=Wij1=W11W21⋮Wr1,i=1,2,⋯,r;j=1,2,⋯,m,W2=Wi2i=1,2,⋯,r=W12,W22,⋯,Wr2Τ,W3=Wi3i=1,2,⋯,r=W13,W23,⋯,Wr3,b1=b11,b21,⋯,br1Τ.So the Jordan neural network is obtained:(5)hit=fi1Wi1Xt+Wi2dt+bi1,1≤t≤T,i=1,2,⋯r,yt=f2∑i=1rWi3hit+b2,1≤t≤T,dt=f3αdt−1+yt−1+b3,1≤t≤T,d0=d1=0.To better understand equation (5), we provide a detailed diagram of the first three steps of the Jordan neural network in Figure 2.Figure 2
Details of Jordan neural network recurrent connections.The following equation is obtained by the Takens theorem of embedding:(6)x^t+1≈yt=gXt,where x^t+1 is the fitting value of the Jordan neural network at time t. And the smooth mapping g can be expressed as in(7)gXt=f2∑i=1rWi3fi1Wi1Xt+Wi2f3αdt−1+yt−1+b3+bi1+b2.


### 4.3. Model for Estimating the Pulse Signal

According to the Takens theorem of embedding, the model of pulse signal, and Jordan neural network, we can obtain the following equation for estimating the pulse signal:(8)xt=st+ct+nt=aΤst+ct+nt,(9)x˜t=xt−aΤst=ct+nt,x˜t+1=gX˜t+εt=gx˜t,x˜t−τ,⋯,x˜t−m−1τ+εt.Then, we can deduce the following equation with unknown parameters involving onlyg⋅ and a:(10)xt+1−aΤst+1=gxt−aΤst,⋯,xt−m−1τ−aΤst−m−1τ+εt.Profile least-square method [34] is a basic idea for fitting a semiparametric model. It is semiparametrically efficient for parametric components, and the nonparametric components are fitted if the parametric components were known. Details of the profile least-square estimation are given in the next section.


#### 4.3.1. Profile Least Square Algorithm

Profile least square algorithm is usually divided into two steps: first, estimating the nonparametric function with a givena, resulting in g^X˜t and second, estimating the unknown parameter a with the estimated function g^X˜t.(A)
Estimation ofgX˜twith Givena.. From equation (10), given the initial value a (null vector), the nonparametric function g^X˜t is obtained by minimizing the residual of fitting:(11)g^X˜t=argmint∑t=n1n−1x˜t+1−gX˜t2,E=12∑t=n1n−1x˜^t+1−x˜t+12.Gradient descent algorithm is used to update the weights of Jordan neural network to achieve the purpose of optimization. The basic idea of gradient descent algorithm is the linear approximation by first-order Taylor expansion. Therefore, the partial derivatives of the weights and the bias that need to be optimized in this paper are, respectively, calculated as follows:(12)∂E∂W3=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,⋯,x^t+1−xt+1f2⋅′hrt,=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,h2t,⋯,hrt,∂E∂W1=∑t=n1n−1x˜^t+1−x˜t+1f2•′W13f11•′XtΤ⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′XtΤ,∂E∂W2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′dt⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′dt,∂E∂b2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′,∂E∂b1=∑t−n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′,∂E∂b3=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi3f3⋅′,∂E∂α=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi2f3⋅′dt−1.The updates of weights and the bias are equal to the sum of the corresponding old one and the product of the partial derivative and the learning rate.(B)
Estimatingafor Giveng^X˜t. For the given g^X˜t, the key to estimate a is to minimize the following equation:(13)a=argmina∑t=n1n−1x˜t+1−g^X˜t2.The method of estimatinga is Newton–Raphson iteration method. Assuming that Ra=∑t=n1n−1x˜t+1−g^X˜t2 and a1 is the minimum solution of Ra, so we have Ra1′=0. Then there is the Taylor expansion at the point a0 near a1:(14)0=Ra1′≈Ra0′+Ra0″a1−a0,a1=a0−Ra0″−1Ra0′.The first derivative and the second derivative ofRa are obtained with respect to the estimated parameters a:(15)Ra′=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a,(16)Ra″=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a−st+1−∂g^X˜t∂aT+2∑t=n1n−1x˜t+1−g^X˜t−∂2g^X˜t∂a2.You obtain the following equation from Sections4.1 through 4.3:(17)X˜t=Xt−Stag^X˜t=f2∑i=1rWi3fi1Wi1X˜t+Wi2f3αdt−1+yt−1+b3+bi1+b2.It can be seen that equation (16) contains the first derivative and the second derivative of g^X˜t with respect to a, and the first derivative is(18)∂g^X˜t∂a1=f2⋅′∑i=1rWi3fi1⋅′Wi1−s1t,⋮∂g^X˜t∂ak=f2⋅′∑i=1rWi3fi1⋅′Wi1−skt.The component form of the first derivative in equation (18) can be written as the following vector form:(19)∂g^X˜t∂a=f2⋅′∑i=1rWi3fi1⋅′Wi1−St.The second derivative ofg^X˜t with respect to a is(20)∂2g^X˜t∂aj2=f2⋅″∑i=1rWi3fi1⋅′Wi1−sjt2+f2⋅′∑i=1rWi3fi1⋅″Wi1−sjt2,j=1,⋯,k.Similarly, the component form of the second derivative in equation (20) can be written as the following vector form:(21)∂2g^X˜t∂a2=f2⋅″∑i=1r∑i=1rWi3fi1⋅′Wi1−StΤWi3fi1⋅′Wi1−St+f2⋅′∑i=1rWi3fi1⋅″Wi1−StΤWi1−St,where St is expressed as(22)St=stΤ,st−τΤ,st−2τΤ,⋯,st−m−1τΤΤ=s1t,s2t,⋯,skt=s1ts2t⋯skts1t−τs2t−τ⋯skt−τ⋮⋮⋮s1t−m−1τs2t−m−1τ⋯skt−m−1τ.


#### 4.3.2. Procedure of Algorithm

We now outline the Algorithm1.Algorithm 1: Jordan neural network estimation algorithm.
Input: Observation sequencext, the initial value of a(a0), precision (0.00001).Output:a1BeginWhile (a1−a0 is greater than the given precision);Step 1: For givena0, update the series xt by equation (9).Step 2: The phase point data setXt is obtained by normalizing and reconstructing xt with the embedding dimension and the time delay.Step 3: Based on (A) in Section4.3.1 above, the Jordan neural network was trained to use the phase point data set Xt.Step 4:a1 is updated using the difference.End whileObtain the optimala1 to achieve the accuracy.EndRemark 2.
Because the mathematical expression of the Jordan neural network is very complex, the difference is used to solve the approximate first and second derivatives for simplifying the calculation when the profile least square is used for estimation. The solution process of difference is as follows:(23)Ra=∑t=n1n−1x˜t+1−gX˜t2=∑t=n1n−1xt+1−aTst+1−gXt−Sta2,Ra′=∂Ra∂a=∂Ra∂a1⋯∂Ra∂akΤ=Ra1−Ra2a1−a2,Ra″=∂2Ra∂a2=Ra1′−Ra3′a1−a3=Ra1−Ra2/a1−a2−Ra3−Ra2/a3−a2a1−a3,=∂2Ra∂a12⋯∂2Ra∂a1∂ak⋮⋱⋮∂2Ra∂ak∂a1⋯∂2Ra∂ak2.
We take 3 points (a1,a2,a3) near a and compute the first derivative Ra′ and the second derivative Ra″ of the function Ra, where Ra′ is a vector of the same form as a. Ra″ is a matrix with k rows and k columns.


## 4.1. Phase Space Reconstruction

Chaotic time series is a univariate or multivariable time series produced by a dynamical system. Phase space reconstruction theorem is mapping a chaotic time series to high-dimensional space. According to the Takens theorem of embedding [32], there is a correlation between one component of the dynamical system and the other components.For an observed chaotic signalxt,t=1,2,⋯,n, the embedding dimension m and the time delay τ could be obtained by using Cao’s method [33], and a phase point in the reconstructed phase space can be expressed as Xt=xt,xt−τ,⋯,xt−m−1τΤ, where t=n1,n1+1,⋯,n and n1=m−1τ+1. Then there is a smooth mapping, g:Rm⟶R, which provides the relationship between Xt and yt, denoted as yt=gXtt=n1,n1+1,⋯,n−1. To predict chaotic time series is to find the function g or the approximate function of g.Remark 1.
In mathematics, Takens embedding theorem gives the conditions under which a chaotic dynamical system can be reconstructed from a sequence of observations of the state of a dynamical system. The reconstruction preserves the properties of the dynamical system that do not change under smooth coordinate changes, but it does not preserve the geometric shape of structures in phase space. It provides the conditions under which a smooth attractor can be reconstructed from the observations made with a generic function.
Cao’s method is a practical method which determines the minimum embedding dimension from a scalar time series. The average ratio of Euclidean distance of a vector with embedded dimensionm to Euclidean distance with embedded dimension m + 1 is denoted as E (m). The ratio of E (m + 1) to E (m) stops changing when m is greater than some value m0 if the time series comes from an attractor. Then m0 + 1 is the minimum embedding dimension we look for.


## 4.2. Jordan Neural Network

Jordan combined the Hopfield network storage concept and distributed parallel processing theory to create a new circular neural network in 1986 [23]. Jordan neural network is made of 4 parts: input layer, hidden layer, output layer, and context layer, respectively. There is a first-order delay operator in the connection from the output layer to the context layer, so that the context layer stores the information of the output layer. There are two types of activation functions in the neural network: linear activation function and nonlinear activation function. Sigmoid nonlinear activation function which is expressed as fx=1/1+e−x is used at the hidden layer and the linear function is used at the output layer in this paper. Jordan neural network topology is shown in Figure 1.Figure 1
Jordan neural network topology.According to Figure1, Xt=xt,xt−τ,xt−2τ,⋯,xt−m−1τΤ is used to represent the input vector of the Jordan neural network, where t=n1,n1+1,⋯,n and n1=1+m−1τ. ht=h1t,h2t,⋯,hrtΤ is the output vector of hidden layer. Wij1i=1,2,⋯,r,j=1,2,⋯,m are the weights from input layer neuron (j) to ith first hidden layer neuron. Wi2i=1,2,⋯,r are the weights from context layer neuron to ith first hidden layer neuron. Wi3i=1,2,⋯,r are the weights from hidden layer neuron (i) to output layer neuron. bi1i=1,2,⋯,r are the biases of hidden layer neuron (i). b2 and b3 are the biases of output layer and context layer, respectively. f2 and f3 are the activation function of output layer and context layer, respectively, and they are usually linear function, while fi1i=1,2,⋯,r are the activation function of hidden layer neuron (i). hiti=1,2,⋯,r is the value of hidden layer neuron (i). dt and yt are the value of context layer and output layer.In order to express clearly and write easily, the following signs will be introduced:(4)W1=Wij1=W11W21⋮Wr1,i=1,2,⋯,r;j=1,2,⋯,m,W2=Wi2i=1,2,⋯,r=W12,W22,⋯,Wr2Τ,W3=Wi3i=1,2,⋯,r=W13,W23,⋯,Wr3,b1=b11,b21,⋯,br1Τ.So the Jordan neural network is obtained:(5)hit=fi1Wi1Xt+Wi2dt+bi1,1≤t≤T,i=1,2,⋯r,yt=f2∑i=1rWi3hit+b2,1≤t≤T,dt=f3αdt−1+yt−1+b3,1≤t≤T,d0=d1=0.To better understand equation (5), we provide a detailed diagram of the first three steps of the Jordan neural network in Figure 2.Figure 2
Details of Jordan neural network recurrent connections.The following equation is obtained by the Takens theorem of embedding:(6)x^t+1≈yt=gXt,where x^t+1 is the fitting value of the Jordan neural network at time t. And the smooth mapping g can be expressed as in(7)gXt=f2∑i=1rWi3fi1Wi1Xt+Wi2f3αdt−1+yt−1+b3+bi1+b2.


## 4.3. Model for Estimating the Pulse Signal

According to the Takens theorem of embedding, the model of pulse signal, and Jordan neural network, we can obtain the following equation for estimating the pulse signal:(8)xt=st+ct+nt=aΤst+ct+nt,(9)x˜t=xt−aΤst=ct+nt,x˜t+1=gX˜t+εt=gx˜t,x˜t−τ,⋯,x˜t−m−1τ+εt.Then, we can deduce the following equation with unknown parameters involving onlyg⋅ and a:(10)xt+1−aΤst+1=gxt−aΤst,⋯,xt−m−1τ−aΤst−m−1τ+εt.Profile least-square method [34] is a basic idea for fitting a semiparametric model. It is semiparametrically efficient for parametric components, and the nonparametric components are fitted if the parametric components were known. Details of the profile least-square estimation are given in the next section.


### 4.3.1. Profile Least Square Algorithm

Profile least square algorithm is usually divided into two steps: first, estimating the nonparametric function with a givena, resulting in g^X˜t and second, estimating the unknown parameter a with the estimated function g^X˜t.(A)
Estimation ofgX˜twith Givena.. From equation (10), given the initial value a (null vector), the nonparametric function g^X˜t is obtained by minimizing the residual of fitting:(11)g^X˜t=argmint∑t=n1n−1x˜t+1−gX˜t2,E=12∑t=n1n−1x˜^t+1−x˜t+12.Gradient descent algorithm is used to update the weights of Jordan neural network to achieve the purpose of optimization. The basic idea of gradient descent algorithm is the linear approximation by first-order Taylor expansion. Therefore, the partial derivatives of the weights and the bias that need to be optimized in this paper are, respectively, calculated as follows:(12)∂E∂W3=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,⋯,x^t+1−xt+1f2⋅′hrt,=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,h2t,⋯,hrt,∂E∂W1=∑t=n1n−1x˜^t+1−x˜t+1f2•′W13f11•′XtΤ⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′XtΤ,∂E∂W2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′dt⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′dt,∂E∂b2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′,∂E∂b1=∑t−n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′,∂E∂b3=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi3f3⋅′,∂E∂α=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi2f3⋅′dt−1.The updates of weights and the bias are equal to the sum of the corresponding old one and the product of the partial derivative and the learning rate.(B)
Estimatingafor Giveng^X˜t. For the given g^X˜t, the key to estimate a is to minimize the following equation:(13)a=argmina∑t=n1n−1x˜t+1−g^X˜t2.The method of estimatinga is Newton–Raphson iteration method. Assuming that Ra=∑t=n1n−1x˜t+1−g^X˜t2 and a1 is the minimum solution of Ra, so we have Ra1′=0. Then there is the Taylor expansion at the point a0 near a1:(14)0=Ra1′≈Ra0′+Ra0″a1−a0,a1=a0−Ra0″−1Ra0′.The first derivative and the second derivative ofRa are obtained with respect to the estimated parameters a:(15)Ra′=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a,(16)Ra″=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a−st+1−∂g^X˜t∂aT+2∑t=n1n−1x˜t+1−g^X˜t−∂2g^X˜t∂a2.You obtain the following equation from Sections4.1 through 4.3:(17)X˜t=Xt−Stag^X˜t=f2∑i=1rWi3fi1Wi1X˜t+Wi2f3αdt−1+yt−1+b3+bi1+b2.It can be seen that equation (16) contains the first derivative and the second derivative of g^X˜t with respect to a, and the first derivative is(18)∂g^X˜t∂a1=f2⋅′∑i=1rWi3fi1⋅′Wi1−s1t,⋮∂g^X˜t∂ak=f2⋅′∑i=1rWi3fi1⋅′Wi1−skt.The component form of the first derivative in equation (18) can be written as the following vector form:(19)∂g^X˜t∂a=f2⋅′∑i=1rWi3fi1⋅′Wi1−St.The second derivative ofg^X˜t with respect to a is(20)∂2g^X˜t∂aj2=f2⋅″∑i=1rWi3fi1⋅′Wi1−sjt2+f2⋅′∑i=1rWi3fi1⋅″Wi1−sjt2,j=1,⋯,k.Similarly, the component form of the second derivative in equation (20) can be written as the following vector form:(21)∂2g^X˜t∂a2=f2⋅″∑i=1r∑i=1rWi3fi1⋅′Wi1−StΤWi3fi1⋅′Wi1−St+f2⋅′∑i=1rWi3fi1⋅″Wi1−StΤWi1−St,where St is expressed as(22)St=stΤ,st−τΤ,st−2τΤ,⋯,st−m−1τΤΤ=s1t,s2t,⋯,skt=s1ts2t⋯skts1t−τs2t−τ⋯skt−τ⋮⋮⋮s1t−m−1τs2t−m−1τ⋯skt−m−1τ.


### 4.3.2. Procedure of Algorithm

We now outline the Algorithm1.Algorithm 1: Jordan neural network estimation algorithm.
Input: Observation sequencext, the initial value of a(a0), precision (0.00001).Output:a1BeginWhile (a1−a0 is greater than the given precision);Step 1: For givena0, update the series xt by equation (9).Step 2: The phase point data setXt is obtained by normalizing and reconstructing xt with the embedding dimension and the time delay.Step 3: Based on (A) in Section4.3.1 above, the Jordan neural network was trained to use the phase point data set Xt.Step 4:a1 is updated using the difference.End whileObtain the optimala1 to achieve the accuracy.EndRemark 2.
Because the mathematical expression of the Jordan neural network is very complex, the difference is used to solve the approximate first and second derivatives for simplifying the calculation when the profile least square is used for estimation. The solution process of difference is as follows:(23)Ra=∑t=n1n−1x˜t+1−gX˜t2=∑t=n1n−1xt+1−aTst+1−gXt−Sta2,Ra′=∂Ra∂a=∂Ra∂a1⋯∂Ra∂akΤ=Ra1−Ra2a1−a2,Ra″=∂2Ra∂a2=Ra1′−Ra3′a1−a3=Ra1−Ra2/a1−a2−Ra3−Ra2/a3−a2a1−a3,=∂2Ra∂a12⋯∂2Ra∂a1∂ak⋮⋱⋮∂2Ra∂ak∂a1⋯∂2Ra∂ak2.
We take 3 points (a1,a2,a3) near a and compute the first derivative Ra′ and the second derivative Ra″ of the function Ra, where Ra′ is a vector of the same form as a. Ra″ is a matrix with k rows and k columns.


## 4.3.1. Profile Least Square Algorithm

Profile least square algorithm is usually divided into two steps: first, estimating the nonparametric function with a givena, resulting in g^X˜t and second, estimating the unknown parameter a with the estimated function g^X˜t.(A)
Estimation ofgX˜twith Givena.. From equation (10), given the initial value a (null vector), the nonparametric function g^X˜t is obtained by minimizing the residual of fitting:(11)g^X˜t=argmint∑t=n1n−1x˜t+1−gX˜t2,E=12∑t=n1n−1x˜^t+1−x˜t+12.Gradient descent algorithm is used to update the weights of Jordan neural network to achieve the purpose of optimization. The basic idea of gradient descent algorithm is the linear approximation by first-order Taylor expansion. Therefore, the partial derivatives of the weights and the bias that need to be optimized in this paper are, respectively, calculated as follows:(12)∂E∂W3=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,⋯,x^t+1−xt+1f2⋅′hrt,=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′h1t,h2t,⋯,hrt,∂E∂W1=∑t=n1n−1x˜^t+1−x˜t+1f2•′W13f11•′XtΤ⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′XtΤ,∂E∂W2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′dt⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′dt,∂E∂b2=∑t=n1n−1x˜^t+1−x˜t+1f2⋅′,∂E∂b1=∑t−n1n−1x˜^t+1−x˜t+1f2⋅′W13f11⋅′⋮x˜^t+1−x˜t+1f2⋅′Wr3fr1⋅′,∂E∂b3=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi3f3⋅′,∂E∂α=∑t=n1n−1∑i=1rx˜^t+1−x˜t+1f2⋅′Wi3fi1⋅′Wi2f3⋅′dt−1.The updates of weights and the bias are equal to the sum of the corresponding old one and the product of the partial derivative and the learning rate.(B)
Estimatingafor Giveng^X˜t. For the given g^X˜t, the key to estimate a is to minimize the following equation:(13)a=argmina∑t=n1n−1x˜t+1−g^X˜t2.The method of estimatinga is Newton–Raphson iteration method. Assuming that Ra=∑t=n1n−1x˜t+1−g^X˜t2 and a1 is the minimum solution of Ra, so we have Ra1′=0. Then there is the Taylor expansion at the point a0 near a1:(14)0=Ra1′≈Ra0′+Ra0″a1−a0,a1=a0−Ra0″−1Ra0′.The first derivative and the second derivative ofRa are obtained with respect to the estimated parameters a:(15)Ra′=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a,(16)Ra″=2∑t=n1n−1x˜t+1−g^X˜t−st+1−∂g^X˜t∂a−st+1−∂g^X˜t∂aT+2∑t=n1n−1x˜t+1−g^X˜t−∂2g^X˜t∂a2.You obtain the following equation from Sections4.1 through 4.3:(17)X˜t=Xt−Stag^X˜t=f2∑i=1rWi3fi1Wi1X˜t+Wi2f3αdt−1+yt−1+b3+bi1+b2.It can be seen that equation (16) contains the first derivative and the second derivative of g^X˜t with respect to a, and the first derivative is(18)∂g^X˜t∂a1=f2⋅′∑i=1rWi3fi1⋅′Wi1−s1t,⋮∂g^X˜t∂ak=f2⋅′∑i=1rWi3fi1⋅′Wi1−skt.The component form of the first derivative in equation (18) can be written as the following vector form:(19)∂g^X˜t∂a=f2⋅′∑i=1rWi3fi1⋅′Wi1−St.The second derivative ofg^X˜t with respect to a is(20)∂2g^X˜t∂aj2=f2⋅″∑i=1rWi3fi1⋅′Wi1−sjt2+f2⋅′∑i=1rWi3fi1⋅″Wi1−sjt2,j=1,⋯,k.Similarly, the component form of the second derivative in equation (20) can be written as the following vector form:(21)∂2g^X˜t∂a2=f2⋅″∑i=1r∑i=1rWi3fi1⋅′Wi1−StΤWi3fi1⋅′Wi1−St+f2⋅′∑i=1rWi3fi1⋅″Wi1−StΤWi1−St,where St is expressed as(22)St=stΤ,st−τΤ,st−2τΤ,⋯,st−m−1τΤΤ=s1t,s2t,⋯,skt=s1ts2t⋯skts1t−τs2t−τ⋯skt−τ⋮⋮⋮s1t−m−1τs2t−m−1τ⋯skt−m−1τ.


## 4.3.2. Procedure of Algorithm

We now outline the Algorithm1.Algorithm 1: Jordan neural network estimation algorithm.
Input: Observation sequencext, the initial value of a(a0), precision (0.00001).Output:a1BeginWhile (a1−a0 is greater than the given precision);Step 1: For givena0, update the series xt by equation (9).Step 2: The phase point data setXt is obtained by normalizing and reconstructing xt with the embedding dimension and the time delay.Step 3: Based on (A) in Section4.3.1 above, the Jordan neural network was trained to use the phase point data set Xt.Step 4:a1 is updated using the difference.End whileObtain the optimala1 to achieve the accuracy.EndRemark 2.
Because the mathematical expression of the Jordan neural network is very complex, the difference is used to solve the approximate first and second derivatives for simplifying the calculation when the profile least square is used for estimation. The solution process of difference is as follows:(23)Ra=∑t=n1n−1x˜t+1−gX˜t2=∑t=n1n−1xt+1−aTst+1−gXt−Sta2,Ra′=∂Ra∂a=∂Ra∂a1⋯∂Ra∂akΤ=Ra1−Ra2a1−a2,Ra″=∂2Ra∂a2=Ra1′−Ra3′a1−a3=Ra1−Ra2/a1−a2−Ra3−Ra2/a3−a2a1−a3,=∂2Ra∂a12⋯∂2Ra∂a1∂ak⋮⋱⋮∂2Ra∂ak∂a1⋯∂2Ra∂ak2.
We take 3 points (a1,a2,a3) near a and compute the first derivative Ra′ and the second derivative Ra″ of the function Ra, where Ra′ is a vector of the same form as a. Ra″ is a matrix with k rows and k columns.


## 5. Experimental Simulations

In order to verify the validity and feasibility of the proposed prediction model, three simulation experiments were carried out. The experimental data in this paper were generated by Lorenz dynamic system and realized byR language. The codes of experiments are uploaded to GitHub website, link is https://github.com/ling-xiu/jordan/blob/master/An application of Jordan neural network. The mean square error(MSE) is used to measure the recovery accuracy of the model:(24)SNRinput=10lgσst2σct2+σnt2,MSE=1n∑t=1nst−s^t2,where(25)σst2=1n∑t=1nst−s¯t2,σct2=1n∑t=1nct−c¯t2.The equation of Lorenz dynamic system is in the following form:(26)x′=σy−x,y′=rx−y−xz,z′=xy−bz,where σ=10,b=8/3,andr=28. Let x, y, and z be the time parameters that are measured according to discretization and the discretization step size is 0.01. We consider the length of data which is used to solve numerical equation (26) by the fourth-order Runge–Kutta method is 10000. We select the first component of equation (26) as the data of chaotic background noise xt. With complex autocorrelation and the method of Cao, the embedding dimension can be determined as m=6 and the delay time can be determined as τ=7.We normalize the original data sets and there are two kinds of normalized equations. Different neural network structures correspond to different normalization equations in this paper:(27)zt=xt−Extσxt,zt=xt−xtminxtmax−xtmin,where xt is the original data and zt is the normalized data sets. Ext is the mean of xt and σxt is the standard deviation of xt.Absolute error (AE) and absolute percentage error (APE) are used to measure the error, and they are(28)AE=a^−a,APE=a^−aa×100%.SNRinput defines the ratio of the variance of st to the variance of the sum of ct and nt.


### 5.1. Example 1: Prediction of Pulse Signals with Different SNRs

Chaotic time seriesct is generated by the Lorenz dynamic system and the index t ranges from 3000 to 7000 for ensuring the chaotic nature of the data. Pulse signal st is generated by equation (2) and the unknown parameters Ti and a are assumed as shown in Table 1. The experimental data consists of ct, st, and white noise sequences nt with a mean of zero.Table 1
Predicted value ofa, AE, and APE with different SNRs.
TiaSNRinputa^AEAPEMSE(1)1300.1−59.181640.09697370.00302630.0302636.9 × 10−8(2)1300.12−57.598020.11618130.00381870.0318231.1 × 10−7(3)1400.15−55.957260.15274670.00274670.0183115.3 × 10−8(4)1400.22−52.630630.22157330.00157330.0071511.7 × 10−8(5)1500.3−50.256320.30053460.00053460.0017821.9 × 10−9(6)1500.4−47.757550.39823380.00176620.0044152.0 × 10−8(7)1500.5−45.819350.49887640.00112360.0022478.2 × 10−9(8)1800.8−42.659030.79419310.00580690.0072591.9 × 10−7(9)2001.1−40.325451.10674870.00674870.0061352.2 × 10−7(10)2001.5−37.631481.50226240.00226240.0015082.4 × 10−8(11)2001.9−35.578231.90663620.00663620.0034932.1 × 10−7(12)2103.6−30.260973.55521660.04478340.0124409.0 × 10−6To demonstrate the stability of the model presented in this paper, we performed Monte Carlo simulations consisting of 50 runs on the same data and the experimental results were averaged.From Table1 and Figure 3, we can see that the convergence rate is very fast, and the iteration times of the 12 sets of data in the experiment can all reach the given accuracy within 20 times. Second, both the absolute error (AE) and the absolute percentage error (APE) of the amplitude of the impulse signal are very small. The mean square error (MSE) of the recovered signal is less than 10−5. As can be seen from Table 2, the model proposed in this paper is very stable. For these 12 sets of data, no difference in the amplitude of the recovered pulse signal was found in either one experiment or 50 Monte Carlo simulation experiments. The standard deviation of the results of 50 Monte Carlo simulation experiments is less than 0.02 and the standard deviation (std) increases with the increase of pulse signal amplitude.Figure 3
Iteration results of the proposed algorithm for Example 1. (1) The iteration result ata = 0.1. (2) The iteration result at a = 0.12. (3) The iteration result at a = 0.15. (4) The iteration result at a = 0.22. (5) The iteration result at a = 0.3. (6) The iteration result at a = 0.4. (7) The iteration result at a = 0.5. (8) The iteration result at a = 0.8. (9) The iteration result at a = 1.1. (10) The iteration result at a = 1.5. (11) The iteration result at a = 1.9. (12) The iteration result at a = 3.6.Table 2
Predicted value ofa and standard deviation with different SNRs.
TiaMean ofa^Standard deviation (std)(1)1300.10.096651550.001731646(2)1300.120.11617270.002553445(3)1400.150.14972190.001991941(4)1400.220.21914950.001827229(5)1500.30.30300180.001105854(6)1500.40.40288390.001321245(7)1500.50.50305580.001121901(8)1800.80.79902410.001515569(9)2001.11.1024120.001636155(10)2001.51.5020590.002687494(11)2001.91.9016250.004017169(12)2103.63.5639630.011684620


### 5.2. Example 2: Prediction Comparison of Different Models

In this experiment, different models were used to fit the same set of observed data, which had an amplitude of 2.5 and a period of 400. The SNR of the observed data is −36.42872. The chaotic noise and original observation signal are shown in Figure4. The models that fit the reconstructed original data are Jordan neural network (JNN), Elman neural network (ENN), backpropagation neural network (BPNN), support vector machine (SVM), and multilayer perceptron (MLP).Figure 4
Data display diagram. (a) Chaotic noisect. (b) Original data xt.
(a)(b)From Table3 and Figure 5, the Jordan neural network has the smallest absolute error between the estimated amplitude and the real value, while the MLP has the largest absolute error. It can be seen from in Figures 6(a), 6(c), and 6(e) that the fitted value of the original signal by Jordan neural network is significantly closer to the real value than support vector machine (SVM) and multilayer perceptron (MLP). It can be seen from the prediction error graph of Figures 6(b), 6(d), and 6(f) that the prediction error of Jordan neural network is relatively large at the time point with pulse signal, and the error value is close to the amplitude of the pulse signal, while the time point error of the rest without pulse signal is relatively small. Support vector machine and multilayer perceptron have large prediction error.Table 3
Predicted value ofa with different models.
ModelaMean ofa^AEAPEstdJNN2.52.5019520.0019520.00078080.004211562BPNN2.52.5032970.0032970.00131880.006069684ENN2.52.5034100.0034100.00136400.005308634SVM2.52.3878660.1121340.04485360.010575260MLP2.52.9696290.4696290.18785160.091993390Figure 5
Comparison diagram of different models.Figure 6
Comparison diagram of original signal and predicted signal and the predicted error diagram: (a) original signal vs. the result of Jordan neural network; (b) Jordan neural network prediction error; (c) original signal vs. the result of support vector machine; (d) support vector machine prediction error; (e) original signal vs. the result of multilayer perceptron; (f) multilayer perceptron prediction error.
(a)(b)(c)(d)(e)(f)


### 5.3. Example 3: Jordan Neural Network Compared with Backpropagation Neural Network

SNRoutput is the signal-to-noise ratio after extracting the pulse signal. SNRoutput defines the ratio of the variance of s^t to the variance of residual:(29)SNRoutput=10lgσs^t2σxt−x^t2.Figure7 takes the input SNR as the horizontal axis and the output SNR as the vertical axis. The experimental data are 12 sets of data from experiment 1, and then the backpropagation neural network estimation is carried out on the basis of experiment 1. It can be seen from Figure 7 that the output SNR increases with the increase of the input SNR, and they are proportional to each other. And with the increase of the input SNR, the increase of the output SNR slows down. It can also be seen from the figure that the output SNR of the Jordan neural network is 10–20db higher than the output SNR of the backpropagation neural network.Figure 7
Output SNR vs. input SNR.


### 5.4. Discussion

This section reports the performance of the profile method combined with Jordan neural network in estimation of pulse signals against chaotic background. The purpose of the experiments was to evaluate if JNN can maintain quality in prediction performance when compared to conventional methods.The results have shown that the method proposed in this paper not only solves the weak pulse signal in chaotic signal well, but also has good stability. We evaluate the application breadth of the proposed model by comparing the experimental data of different SNR. Table1 shows that the estimated SNR range of the proposed model is −60 to −30. Within this range, the model proposed in this paper can effectively estimate the pulse signal amplitude, and its absolute error (AE) can be as low as 0.00112 and as high as 0.05, and its MSE is also below 10−5.The results report the estimation of pulse signals by different neural networks under chaotic noise background, as well as the mean amplitude and standard deviation (std) of 50 test runs in Table3. We can also see from Table 3 that the two minimum standard deviations are 0.00421 and 0.00531, which are, respectively, from the Jordan neural network and the Elman neural network. We know that both the Jordan neural network and the Elman neural network are feedback neural networks. The difference between them is that the context layer connects to a different layer. Compared with [35], we found that the Jordan neural network could better fit the time series than the Elman neural network.A major innovation of this method is to use Jordan neural network to fit the chaotic noise background and then estimate the impulse signal in the chaotic noise by combining the profile method. Compared with other neural networks, this method is more accurate and the overall training time is shorter.


## 5.1. Example 1: Prediction of Pulse Signals with Different SNRs

Chaotic time seriesct is generated by the Lorenz dynamic system and the index t ranges from 3000 to 7000 for ensuring the chaotic nature of the data. Pulse signal st is generated by equation (2) and the unknown parameters Ti and a are assumed as shown in Table 1. The experimental data consists of ct, st, and white noise sequences nt with a mean of zero.Table 1
Predicted value ofa, AE, and APE with different SNRs.
TiaSNRinputa^AEAPEMSE(1)1300.1−59.181640.09697370.00302630.0302636.9 × 10−8(2)1300.12−57.598020.11618130.00381870.0318231.1 × 10−7(3)1400.15−55.957260.15274670.00274670.0183115.3 × 10−8(4)1400.22−52.630630.22157330.00157330.0071511.7 × 10−8(5)1500.3−50.256320.30053460.00053460.0017821.9 × 10−9(6)1500.4−47.757550.39823380.00176620.0044152.0 × 10−8(7)1500.5−45.819350.49887640.00112360.0022478.2 × 10−9(8)1800.8−42.659030.79419310.00580690.0072591.9 × 10−7(9)2001.1−40.325451.10674870.00674870.0061352.2 × 10−7(10)2001.5−37.631481.50226240.00226240.0015082.4 × 10−8(11)2001.9−35.578231.90663620.00663620.0034932.1 × 10−7(12)2103.6−30.260973.55521660.04478340.0124409.0 × 10−6To demonstrate the stability of the model presented in this paper, we performed Monte Carlo simulations consisting of 50 runs on the same data and the experimental results were averaged.From Table1 and Figure 3, we can see that the convergence rate is very fast, and the iteration times of the 12 sets of data in the experiment can all reach the given accuracy within 20 times. Second, both the absolute error (AE) and the absolute percentage error (APE) of the amplitude of the impulse signal are very small. The mean square error (MSE) of the recovered signal is less than 10−5. As can be seen from Table 2, the model proposed in this paper is very stable. For these 12 sets of data, no difference in the amplitude of the recovered pulse signal was found in either one experiment or 50 Monte Carlo simulation experiments. The standard deviation of the results of 50 Monte Carlo simulation experiments is less than 0.02 and the standard deviation (std) increases with the increase of pulse signal amplitude.Figure 3
Iteration results of the proposed algorithm for Example 1. (1) The iteration result ata = 0.1. (2) The iteration result at a = 0.12. (3) The iteration result at a = 0.15. (4) The iteration result at a = 0.22. (5) The iteration result at a = 0.3. (6) The iteration result at a = 0.4. (7) The iteration result at a = 0.5. (8) The iteration result at a = 0.8. (9) The iteration result at a = 1.1. (10) The iteration result at a = 1.5. (11) The iteration result at a = 1.9. (12) The iteration result at a = 3.6.Table 2
Predicted value ofa and standard deviation with different SNRs.
TiaMean ofa^Standard deviation (std)(1)1300.10.096651550.001731646(2)1300.120.11617270.002553445(3)1400.150.14972190.001991941(4)1400.220.21914950.001827229(5)1500.30.30300180.001105854(6)1500.40.40288390.001321245(7)1500.50.50305580.001121901(8)1800.80.79902410.001515569(9)2001.11.1024120.001636155(10)2001.51.5020590.002687494(11)2001.91.9016250.004017169(12)2103.63.5639630.011684620


## 5.2. Example 2: Prediction Comparison of Different Models

In this experiment, different models were used to fit the same set of observed data, which had an amplitude of 2.5 and a period of 400. The SNR of the observed data is −36.42872. The chaotic noise and original observation signal are shown in Figure4. The models that fit the reconstructed original data are Jordan neural network (JNN), Elman neural network (ENN), backpropagation neural network (BPNN), support vector machine (SVM), and multilayer perceptron (MLP).Figure 4
Data display diagram. (a) Chaotic noisect. (b) Original data xt.
(a)(b)From Table3 and Figure 5, the Jordan neural network has the smallest absolute error between the estimated amplitude and the real value, while the MLP has the largest absolute error. It can be seen from in Figures 6(a), 6(c), and 6(e) that the fitted value of the original signal by Jordan neural network is significantly closer to the real value than support vector machine (SVM) and multilayer perceptron (MLP). It can be seen from the prediction error graph of Figures 6(b), 6(d), and 6(f) that the prediction error of Jordan neural network is relatively large at the time point with pulse signal, and the error value is close to the amplitude of the pulse signal, while the time point error of the rest without pulse signal is relatively small. Support vector machine and multilayer perceptron have large prediction error.Table 3
Predicted value ofa with different models.
ModelaMean ofa^AEAPEstdJNN2.52.5019520.0019520.00078080.004211562BPNN2.52.5032970.0032970.00131880.006069684ENN2.52.5034100.0034100.00136400.005308634SVM2.52.3878660.1121340.04485360.010575260MLP2.52.9696290.4696290.18785160.091993390Figure 5
Comparison diagram of different models.Figure 6
Comparison diagram of original signal and predicted signal and the predicted error diagram: (a) original signal vs. the result of Jordan neural network; (b) Jordan neural network prediction error; (c) original signal vs. the result of support vector machine; (d) support vector machine prediction error; (e) original signal vs. the result of multilayer perceptron; (f) multilayer perceptron prediction error.
(a)(b)(c)(d)(e)(f)


## 5.3. Example 3: Jordan Neural Network Compared with Backpropagation Neural Network

SNRoutput is the signal-to-noise ratio after extracting the pulse signal. SNRoutput defines the ratio of the variance of s^t to the variance of residual:(29)SNRoutput=10lgσs^t2σxt−x^t2.Figure7 takes the input SNR as the horizontal axis and the output SNR as the vertical axis. The experimental data are 12 sets of data from experiment 1, and then the backpropagation neural network estimation is carried out on the basis of experiment 1. It can be seen from Figure 7 that the output SNR increases with the increase of the input SNR, and they are proportional to each other. And with the increase of the input SNR, the increase of the output SNR slows down. It can also be seen from the figure that the output SNR of the Jordan neural network is 10–20db higher than the output SNR of the backpropagation neural network.Figure 7
Output SNR vs. input SNR.


## 5.4. Discussion

This section reports the performance of the profile method combined with Jordan neural network in estimation of pulse signals against chaotic background. The purpose of the experiments was to evaluate if JNN can maintain quality in prediction performance when compared to conventional methods.The results have shown that the method proposed in this paper not only solves the weak pulse signal in chaotic signal well, but also has good stability. We evaluate the application breadth of the proposed model by comparing the experimental data of different SNR. Table1 shows that the estimated SNR range of the proposed model is −60 to −30. Within this range, the model proposed in this paper can effectively estimate the pulse signal amplitude, and its absolute error (AE) can be as low as 0.00112 and as high as 0.05, and its MSE is also below 10−5.The results report the estimation of pulse signals by different neural networks under chaotic noise background, as well as the mean amplitude and standard deviation (std) of 50 test runs in Table3. We can also see from Table 3 that the two minimum standard deviations are 0.00421 and 0.00531, which are, respectively, from the Jordan neural network and the Elman neural network. We know that both the Jordan neural network and the Elman neural network are feedback neural networks. The difference between them is that the context layer connects to a different layer. Compared with [35], we found that the Jordan neural network could better fit the time series than the Elman neural network.A major innovation of this method is to use Jordan neural network to fit the chaotic noise background and then estimate the impulse signal in the chaotic noise by combining the profile method. Compared with other neural networks, this method is more accurate and the overall training time is shorter.


## 6. Conclusions

In this paper, we are interested in weak pulse signal in chaotic background. Based on the short-term predictability and Takens theorem, we provide an algorithm for directly estimating the problems; that is, the Jordan neural network is used to fit the chaotic time series, and the one-step prediction error was obtained. Then, starting from the error, the single-point jump signal model was connected, and the amplitude of the pulse signal was estimated by the profile least square method, so as to achieve the prediction effect of the pulse signal under the chaotic background. The following conclusions can be drawn from the experimental results: the model proposed in this paper can predict the weak pulse signal in the chaotic background, and it can be seen from the results of experiment 1 that the prediction accuracy is high. It can be seen from the results of experiment 2 that the Jordan neural network is significantly better than other feed-forward neural networks in fitting the nonlinear dynamic system, and the absolute percentage error of the Jordan neural network is the smallest. In future work, we can try to improve in two aspects, so that it has the best generalization performance in most cases, and apply the method to solve the relevant practical problems. Firstly, we can try to estimate the amplitude of the impulse signal without knowing the period of the impulse signal. Secondly, the Jordan neural network can be trained by other optimization methods, and the Jordan neural network can be improved.



---

*Source: 3284587-2020-07-20.xml*