# An Intelligent Calculation Method of Volterra Time-Domain Kernel Based on Time-Delay Artificial Neural Network

Authors: Kaiyang Zhong; Lerui Chen
Journal: Mathematical Problems in Engineering
 (2020)

---

## Abstract

To solve the problems of high complexity and low accuracy in Volterra time-domain kernel calculation of a nonlinear system, this paper proposes an intelligent calculation method of Volterra time-domain kernel by time-delay artificial neural networks (TDANNs) and also designs a root mean square error (RMSE) index to choose the neuron number of the network input layer. Firstly, a three-layer TDANN is designed according to the characteristics of the Volterra model. Secondly, the relationship between parameters of TDANN and Volterra time-domain kernel is analyzed, and then three-order expressions of Volterra time-domain kernel are derived. The calculation of Volterra time-domain kernel is completed by network training. Finally, it is verified by a nonlinear system. Simulation results indicate that compared with traditional methods, the new method has higher accuracy, and it can realize the batch calculation of Volterra kernel, which not only improves the calculation efficiency but also provides accurate data for fault diagnosis based on Volterra kernel in further research work.


---

## Body

## 1. Introduction

As the transfer function of a linear system, Volterra kernel (including Volterra kernel time-domain kernel and Volterra kernel frequency-domain kernel) does not depend on the input and output of the system, which can characterize the essential properties of the nonlinear system and explain the special phenomena of the nonlinear system. It has advantages of clear physical meaning and rich information, which is widely used in electronic engineering, mechanical and electrical engineering, control engineering, and other fields [1–6].At present, the research and application of Volterra kernel have developed rapidly; especially the nonlinear spectrum research based on the generalized frequency response function (GFRF) and the output spectrum response function (OFRF) has been gradually applied to the fields of feature extraction, mechanism analysis, and fault diagnosis [7–11]; however, the complexity and accuracy of Volterra kernel calculation are the key factors to its application. Until now, the calculation of Volterra kernel is still difficult, and the main calculation methods of Volterra kernel are the recursive method and identification method. The recursive algorithm is to substitute the relevant parameters of the system into the recursive equation to calculate the Volterra kernel. Billings and Tsang [12] adopted recursive algorithm to obtain Volterra kernels of each order for a nonlinear system. Jing et al. [13] established a description function from low-order Volterra kernel to high-order Volterra kernel to calculate the Volterra kernel of each order based on the nonlinear differential equation. Billings and Peyton Jones [14] adopted the harmonic probing method to derive the recursive algorithm of Volterra frequency-domain kernel. Swain and Billings [15] adopted recursive algorithm to calculate the Volterra kernel of an MIMO system. The research studies mentioned above calculated the Volterra kernel based on a system mathematical model, and the calculation accuracy depends entirely on the accuracy of the model. Obviously, recursive algorithm is not suitable for the system with uncertain mathematical model. The identification algorithm is a “black box” operation, which does not rely on the mathematical model of the system; it only relies on the input and output data of the system. The Volterra kernel of each order can be calculated by adjusting the parameters according to the error between the true output and the estimated output. Toker and Emara-Shabaik [16] proposed LMS identification algorithm by using pseudorandom multilevel sequence excitation, which can avoid matrix inversion and improve calculation speed and accuracy. Sigrist et al. [17] adopted the improved LMS method to identify the first two-order Volterra time-domain kernel. Asyali and Juusola [18] expanded the Volterra kernel by the Meixner function to identify the Volterra model of a nonlinear system with time delay. Prazenica and Kurdila [19] proposed multiwavelet function to identify Volterra time-domain kernel, and the first three-order Volterra time-domain kernel function of a nonlinear system was obtained. The identification algorithm can realize the adaptive calculation, but this method calculated one Volterra kernel value at one time, which cannot achieve batch calculation and cannot meet the needs of fault diagnosis based on data. Therefore, there are at least two shortcomings in the traditional calculation of Volterra kernel: (1) the calculation efficiency and process are low and complex, respectively; (2) due to the approximate processing in the calculation process, the error between the estimated value and the true value is large.In order to solve the problems of calculation methods mentioned above, a new intelligent calculation method is urgently needed. Artificial neural network (ANN) is a parallel distributed information processing network. The neurons in the network have strong ability of nonlinear approximation and fitting, which can approach any nonlinear function [20–22]. In recent years, to simplify Volterra kernel calculation, a small number of researchers have tried to adopt ANN to calculate the Volterra kernel. Wray and Green [23] and Stegmayer et al. [24] adopted the weight coefficient of ANN to express the Volterra kernel of each order, and it was also verified by a nonlinear system. In references [25, 26], a Volterra kernel identification method based on time-delay neural network (TDANN) was proposed, and the relationship between the parameters of network and Volterra kernel function was derived, which successfully identified the first three-order Volterra kernel of nonlinear unsteady aerodynamic loading. Although ANN adopted in above research studies can realize fast and accurate calculation of Volterra kernel, there are still some disadvantages, for example, the number of input nodes in the network structure was set by experience, and the analysis of input node number impact on the calculation accuracy was lacking; furthermore, the network structure was established with great randomness and arbitrariness, which has no adaptability.The main contributions of this paper are as follows. (1) Based on the research studies by the authors of [23–26], the impact of input node number and the memory length of TDANN network on the accuracy and calculation speed is analyzed and studied. (2) Root mean square error (RMSE) is introduced as the reference of neuron node number selection to realize network structure adaptive adjustment, which can avoid the network structure selection blindness. After obtaining the optimal structure, only the input and output of the system were needed to realize the intelligent calculation of the Volterra kernel. (3) The proposed method was verified by a nonlinear system. The reason why it was called “intelligent calculation” is that compared with the traditional calculation method, the whole calculation process does not need human’s participation, which can avoid the complex calculation process. In Section 2, the mathematical model of Volterra time-domain kernel is provided. Section 3 introduces the concept of TDANN. In Section 4, the relationship between the Volterra kernel model and TDANN model is analyzed, and the explicit expression of Volterra time-domain kernel is obtained. In Section 5, the proposed method is simulated and verified by a nonlinear model. The conclusions are summarized in Section 6.


## 2. Volterra Time-Domain Kernel Model

For any continuous time invariant nonlinear system, if the system input satisfies(1)∫−∞+∞u2t<+∞,then, the output of system can be expressed by Volterra series [27–29], namely,(2)yt=∑n=1+∞ynt,(3)ynt=∫−∞+∞⋯∫−∞+∞hnτ1,τ2,…,τn∏i=1nut−τidτi,whereyt is output of the system, ut is input of the system, and hnτ1,τ2,…,τn is the n-order Volterra time-domain kernel or generalized frequency response function. The n-order Volterra kernel model can be obtained by truncating (2), which is shown as(4)yt=h0+∫0+∞h1τ1ut−τ1dτ1+∫0+∞∫0+∞h2τ1,τ2ut−τ1ut−τ2dτ1dτ2+⋯+∫0+∞∫0+∞∫0+∞hnτ1,τ2,…,τnut−τ1ut−τ2…ut−τndτ1dτ2…dτn.Discretize equation (3) to obtain the discrete Volterra kernel model, which is shown as(5)yk=h0+∑m1=0+∞h1m1uk−m1+∑m1=0+∞∑m2=0+∞h2m1,m2uk−m1uk−m2+⋯+∑m1=0+∞∑m2=0+∞…∑mn=0+∞hnm1,m2,…,mnuk−m1uk−m2.For a system with memory lengthM, equation (4) can be changed to(6)yk=h0+∑m1=0Mh1m1uk−m1+∑m1=0M∑m2=0Mh2m1,m2uk−m1uk−m2+⋯+∑m1=0M∑m2=0M⋯∑mn=0Mhnm1,m2,…,mnuk−m1uk−m2⋯uk−mn.Generally speaking, the energy components of Volterra kernels above three-order are very small, and it is also not easy to visualize the graphs over three dimensions [30]. Therefore, this paper only calculates the first three-order Volterra kernel of the system.


## 3. Time-Delay Artificial Neural Networks (TDANNs)

As shown in Figure1, the artificial neural network is composed of many neurons which are related to each other to achieve efficient data processing. Such a network is also known as feedforward multilayer perceptron. Artificial neural network is generally composed of an input layer, hidden layer, and output layer. A simple three-layer network can realize the approximation of any nonlinear function. A mapping of data from the input layer to the output layer is called training. In order to achieve accurate approximation, several training iterations are required. In this process, the network constantly updates the weights and thresholds between different neurons. According to the error between true value and calculation result, the weights and thresholds are modified in turn. After updating several times, the calculation result is closer to the true value, and the optimal weight and threshold can also be obtained.Figure 1
Artificial neural network with single hidden layer.ANN is a typical MIMO neural network withM inputs and N outputs. The selection of input and output of a neural network depends on the characteristics of the specific system. For the Volterra kernel model, from equation (5), it can be seen that the Volterra kernel model with memory length M has M + 1 input variables, which are uk,uk−1,⋯uk−M, respectively. There is one output, that is yk. In addition, hk−1,…,uk−M are delayed 1 t, 2 t,..., M t, respectively, from input uk. For a single-input single-output (SISO) system, how to construct dataset with M+1 inputs and 1 output is important. The specific process is shown in Figure 2.Figure 2
The process of dataset construction.The artificial neural network takesM+1 delay neurons as the input layer and 1 output neuron as the output layer, which is called time-delay artificial neural network (TDANN). Because the network adopts forward time delay, it has the function of memory. The structure of TDANN is shown in Figure 3.Figure 3
The structure of TDANN.In Figure3, ω11,φ12,…,ωMp are weights between the input layer and hidden layer; b1,b2,…,bp are thresholds on the hidden layer; ω1,ω2,…,ωp are weights between the hidden layer and output layer; and b0 is the threshold on the output layer.


## 4. Equation Derivation of Volterra Kernel Based on TDANN

According to equation (3), the first three-order Volterra kernel model with memory length M can be shown as(7)yk=h0+∑m1=0Mh1m1uk−m1+∑m1=0M∑m2=0Mh2m1,m2uk−m1uk−m2+∑m1=0M∑m2=0M∑m3=0Mhnm1,m2,m3uk−m1uk−m2uk−m3.For the TDANN model shown in Figure3, the selection of activation function and number of neurons in the hidden layer directly affects the accuracy of output. In this paper, sigmoid function is chosen as the activation function of neurons in the hidden layer, which is shown as(8)φx=11+e−x.The number of hidden neurons has a significant impact on the performance of the network. If the number of neurons is too small, the calculation accuracy is low. If the number of neurons is too large, the overfitting phenomena will occur. So, according to the Volterra kernel characteristics, this paper adopts root mean square error (RMSE) as an evaluation index to determine the reasonable number of hidden neurons, which is shown as(9)RMSE=1k∑j=1kyj−y^j2,whereyj is the true value of output at time j; y^j is the predicted value of output at time j; and RMSE represents the standard deviation of error samples between the estimated value and the true value, which reflects the dispersion degree of sample. The smaller the RMSE is, the better the approximation effect is.φ
x is chosen as activation function between neurons in the input layer and hidden layer, and pureline function is chosen as activation function between neurons in the hidden layer and output layer; then, the j-th output yj can be expressed as(10)yj=φ∑i=1M+1ωij∗ui+bj,where M is the memory length of the network; ωij is the weight between the i-th neuron in the input layer and the j-th neuron in the hidden layer; ui is the i-th input; and bj is the threshold of j-th neuron in the hidden layer.The output of TDANN can be expressed by(11)yout=∑j=1pωj∗yj+b0,where p is the number of neurons, ωj is the weight between the j-th neuron in the hidden layer and output layer, and b0 is the threshold on the output layer. Substituting equation (9) into equation (10) and expanding φxwith Taylor series, equation (12) can be obtained.(12)yout=∑j=1pωj∗∑n=0+∞φnbjn!∗∑i=1M+1ωij∗uk−i−1n+b0,where φnbjstands for the n-th derivative of φx at bj. Because only the first three-order Volterra kernel values are considered in this paper, equation (11) can be translated into(13)yout=∑j=1pωj∗∑n=03φnbjn!∗∑i=1M+1ωij∗uk−i−1n+b0.Expandingn and i in equation (13), equation (14) can be obtained.(14)yout=b0+∑j=1pωj∗φbj+∑j=1pωj∗φ1bjω1j∗uk+ω2j∗uk−1+⋯+ωM+1j∗uk−M+∑j=1pωj∗φ2bj2!ω1j∗uk+ω2j∗uk−1+⋯+ωM+1j∗uk−M2+∑j=1pωj∗φ3bj3!ω1j∗uk+ω2j∗uk−1+…+ωM+1j∗uk−M3.Comparing equation (6) and equation (14), the first three-order Volterra time-domain kernel calculation equations can be obtained, which are shown as follows:(15)h0=b0+∑j=1pωj∗φbj,(16)h1m=∑j=1pωj∗ωm+1j∗φ1bjm=0,1,2,…,M ,(17)h2m1,m2m1≠m2=∑j=1pωj∗ωm1+1j∗ωm2+1j∗φ2bjm1=0,1,2,…,M;m2=0,1,2,…,M ,(18)h2m1,m2m1=m2=∑j=1pωj∗ωm1+1j2∗φ2bj2!m1=0,1,2,…,M;m2=0,1,2,…,M,(19)h3m1,m2,m3m1≠m2≠m3=∑j=1pωj∗ωm1+1j∗ωm2+1j∗ωm3+1j∗φ3bjm1=0,1,2,…,M;m2=0,1,2,…,M;m3=0,1,2,…,M ,(20)h3m1,m2,m3m1=m2=m3=∑j=1pωj∗ωm1+1j3∗φ3bj3!m1=0,1,2,…,M;m2=0,1,2,…,M;m3=0,1,2,…,M ,(21)h3m1,m2,m3m1=m3≠m2m1=m2≠m3m1≠m2=m3=∑j=1pωj∗ωm1+1j∗ωm2+1j∗ωm3+1j∗φ3bj2m1=0,1,2,…,M;m2=0,1,2,…,M;m3=0,1,2,…,M.It can be seen from equations (15)–(21) that the first three-order Volterra time-domain kernel expressions are completely composed of weights and thresholds of TDANN. We substitute the optimal weights and thresholds into the equations above to calculate the Volterra kernel of each order. Compared with the traditional calculation method, the new method only needs the input and output of system to train the network and does not need manual participation, which can realize the intelligent calculation of Volterra kernel.


## 5. Simulations

According to reference [31], the dynamic model of the nonlinear oscillator is shown in Figure 4.Figure 4
The model of the nonlinear oscillator.The nonlinear differential equation of the model can be expressed as follows:(22)md2ytdt2+cdytdt+k1yt+k2y2t+k3y3t=ut,where m is the mass of the oscillator; c is the damping coefficient of the system; k1 and k2 are linear stiffness parameters and k3 is nonlinear stiffness parameter; ut is external force applied to the system, which is the input of the system; and yt is the displacement of the vibrator, which is the output of the system.The parameters are(23)m1=1kg,c=3N⋅sm,k1=60Nm,k2=100Nm2,k3=200Nm3,setting the input as ut=50cos0.4πt.


### 5.1. The Effect of Volterra Memory LengthM on System Output

From Figures2 and 3, it can be seen that the memory length M not only affects the neuron number of the input layer but also affects the composition of dataset. If M is too small, the nonlinear correlation between neurons is not high. If M is too large, overfitting phenomena may occur. So, it is very important to choose M reasonably to ensure the output accuracy. In order to facilitate the analysis, this paper makes the neuron number of the input layer equal to the hidden layer, namely, M + 1 = p. The influences of M on the network output are tested, which are shown in Figures 5–7 and Table 1.Figure 5
M = 3: the comparison of output between true value and estimated value.Figure 6
M = 8: the comparison of output between true value and estimated value.Figure 7
M = 10: the comparison of output between true value and estimated value.Table 1
The effects ofM on system output.
M
Average error
RMSE
Time cost (s)
2
0.0013
0.006
0.91
3
6.3360 × 10−4
0.0059
0.85
4
8.5099 × 10−4
0.0051
0.90
5
6.1712 × 10−4
0.0042
0.79
6
0.0033
0.0052
0.96
7
1.3466 × 10−4
0.0052
0.89
8
8.7882 × 10
−5
0.0013
1.02
9
8.3969 × 10−4
0.0031
1.24
10
0.0021
0.0050
1.29The specific effects of memory lengthM on system output are shown in Table 1.From Figures5–7 and Table 1, it can be seen that M has a great effect on the accuracy of system output. At the beginning, when M is gradually increasing, the system output error tends to decrease due to the increase of correlation degree between neurons in the network. However, with the increase of M, the degree of correlation between neurons is increasing, but the overfitting phenomena occurred, so the system output error shows oscillation. In addition, with the increase of M, the structure of network becomes more complex, and the time cost of network training and testing is longer. Furthermore, based on the RMSE index, the memory length M is selected to be 8.


### 5.2. The Calculation of Volterra Kernel

WhenM = 8, the neuron number of the input layer is 9, and the neuron number of the hidden layer is 9. After TDANN training, the optimal weight of different neurons between the input layer and hidden layer is(24)ωi,j=0.49910.43270.45830.42400.40350.33510.29060.31920.28280.01160.01180.01990.04430.03490.0092−0.0498−0.0302−0.0562−0.2804−0.3477−0.3369−0.3534−0.4014−0.3974−0.4240−0.4185−0.4601−0.1466−0.1077−0.04150.03620.05560.11750.15560.16160.20203.65852.66031.68660.7550−0.2669−1.2297−2.2024−3.1543−4.0925−0.7376−0.5524−0.3880−0.17660.01060.19920.35930.57230.70830.27530.23060.16390.07040.0402−0.0262−0.1491−0.1554−0.24140.03400.08280.00360.00950.0146−0.0228−0.0451−0.0031−0.06560.06490.02330.0389−0.0084−0.0120−0.0324−0.01950.01400.0064.The optimal threshold of neurons in the hidden layer is(25)b=−2.42542.6970−2.41431.05300.00960.78091.9744−2.73373.5764.The optimal weight of different neurons between the hidden layer and output layer is(26)ωj=0.3457−0.0424−0.3568−0.1451−1.8627−1.18510.15610.68170.4550.The optimal threshold of neurons in the output layer is(27)b0=1.2744.The first three-order Volterra time-domain kernel can be obtained by substituting the parameters above into equations (15)–(20), and the results are shown in Figures 8–10.Figure 8
The first-order Volterra time-domain kernel calculated by TDANN.Figure 9
The second-order Volterra time-domain kernel calculated by TDANN.Figure 10
The third-order Volterra time-domain kernel calculated by TDANN (m3=0.In order to prove that the identified kernels hold the correct dynamics, predicted responses can be obtained by the first three-order Volterra time-domain kernels, which are shown in Figure11.Figure 11
The predicted response of each assessed kernel.From Figure11, it can be seen that the predicted response using first-order kernel is similar to the system input, which stands for the linear system response, and the predicted response using first two-order kernels begin to appear as nonlinearity components; furthermore, with the order of kernel increasing, the nonlinear components are more and more obvious and also close to the real response.


### 5.3. The Robustness of the Proposed Method

In order to demonstrate the robustness of the proposed method, Gaussian noise is added to the system input; the first three-order Volterra kernels are calculated by TDANN, and then the system output estimated value can be obtained by equation (6), which is shown in Figure 12.Figure 12
The robustness of the proposed method.From Figure12, it can be seen that due to the influence of noise, the estimated output value of the system is in fluctuation state at beginning, and the errors between the estimated value and the true value are relatively large, but after about 2 s, the estimated value begins to approach the true value, and after 4.7 s, the error between the estimated value and the true value is getting smaller and smaller. So, it can be demonstrated that the proposed method has robustness.


### 5.4. The Accuracy Comparison with Traditional Algorithm

In order to verify the accuracy of Volterra kernel calculated by TDANN, the first three-order Volterra time-domain kernels of the nonlinear system are calculated by TDANN, recursive algorithm in reference [32], and variable step size least mean square (VSSLMS) adaptive identification algorithm in reference [33], respectively. Then, three different output results are obtained by equation (6). The accuracy of three methods is shown in Figures 13 and 14.Figure 13
The output comparison of three methods.Figure 14
The errors of three methods.From Figures13 and 14, it can be seen that among three methods, the accuracy of the TDANN method is highest, so each order of Volterra time-domain kernel (as shown in Figures 7–9) calculated by TDANN is closest to the true value of Volterra time-domain kernel. In the VSSLMS identification algorithm, all kinds of parameters have been adjusted to the best, but the identification accuracy is not ideal for the system; two reasons can account for such phenomena. Firstly, the adjustment of parameters in the identification algorithm is based on experience, which is subjective. Secondly, the accuracy of the identification algorithm may be affected by the data type. So, the accuracy of identification algorithm is the lowest. It is worth to notice that in order to further enhance the accuracy, TDANN can easily realize the above three-order Volterra time-domain kernel calculation, while it is difficult for others to calculate the Volterra time-domain kernel above three-order.


### 5.5. The Computation Complexity Comparison with Traditional Algorithm

Both TDANN and recursive algorithm can realize the simultaneous calculation of multiple Volterra kernels, namely, one operation can realize the calculation of multiple Volterra kernels, which can improve the calculation speed and efficiency, while with identification algorithm, one operation can only calculate one Volterra kernel value. In order to demonstrate that TDANN can reduce the computational complexity, the recursive algorithm in reference [32] and the identification algorithm in reference [33] are adopted to randomly calculate several first three-order Volterra kernels of the system. The time consumption of three methods is shown in Table 2.Table 2
The time consumption of different methods.
Samples
Identification algorithm (ms)
Recursive algorithm (ms)
TDANN (ms)
2
113.34
185.69
145.31
4
183.47
6
124.28
(2,3)
129.35
194.16
117.48
(4,7)
175.48
(5,5)
116.72
(1,4,0)
163.89
179.35
104.17
(4,7,0)
142.51
(5,6,0)
176.19It can be seen that the identification algorithm consumed the most time, and TDANN consumed the least time. The time consumption of identification algorithm, recursive algorithm, and TDANN is 1325.25 ms, 559.2 ms, and 366.96 ms, respectively. Compared with identification algorithm and recursive algorithm, the calculation speed of TDANN is increased by 72.31% and 34.38%, respectively. Therefore, the method proposed in this paper reduced the complexity of calculation, which can realize the online calculation and detection of the system.


## 5.1. The Effect of Volterra Memory LengthM on System Output

From Figures2 and 3, it can be seen that the memory length M not only affects the neuron number of the input layer but also affects the composition of dataset. If M is too small, the nonlinear correlation between neurons is not high. If M is too large, overfitting phenomena may occur. So, it is very important to choose M reasonably to ensure the output accuracy. In order to facilitate the analysis, this paper makes the neuron number of the input layer equal to the hidden layer, namely, M + 1 = p. The influences of M on the network output are tested, which are shown in Figures 5–7 and Table 1.Figure 5
M = 3: the comparison of output between true value and estimated value.Figure 6
M = 8: the comparison of output between true value and estimated value.Figure 7
M = 10: the comparison of output between true value and estimated value.Table 1
The effects ofM on system output.
M
Average error
RMSE
Time cost (s)
2
0.0013
0.006
0.91
3
6.3360 × 10−4
0.0059
0.85
4
8.5099 × 10−4
0.0051
0.90
5
6.1712 × 10−4
0.0042
0.79
6
0.0033
0.0052
0.96
7
1.3466 × 10−4
0.0052
0.89
8
8.7882 × 10
−5
0.0013
1.02
9
8.3969 × 10−4
0.0031
1.24
10
0.0021
0.0050
1.29The specific effects of memory lengthM on system output are shown in Table 1.From Figures5–7 and Table 1, it can be seen that M has a great effect on the accuracy of system output. At the beginning, when M is gradually increasing, the system output error tends to decrease due to the increase of correlation degree between neurons in the network. However, with the increase of M, the degree of correlation between neurons is increasing, but the overfitting phenomena occurred, so the system output error shows oscillation. In addition, with the increase of M, the structure of network becomes more complex, and the time cost of network training and testing is longer. Furthermore, based on the RMSE index, the memory length M is selected to be 8.


## 5.2. The Calculation of Volterra Kernel

WhenM = 8, the neuron number of the input layer is 9, and the neuron number of the hidden layer is 9. After TDANN training, the optimal weight of different neurons between the input layer and hidden layer is(24)ωi,j=0.49910.43270.45830.42400.40350.33510.29060.31920.28280.01160.01180.01990.04430.03490.0092−0.0498−0.0302−0.0562−0.2804−0.3477−0.3369−0.3534−0.4014−0.3974−0.4240−0.4185−0.4601−0.1466−0.1077−0.04150.03620.05560.11750.15560.16160.20203.65852.66031.68660.7550−0.2669−1.2297−2.2024−3.1543−4.0925−0.7376−0.5524−0.3880−0.17660.01060.19920.35930.57230.70830.27530.23060.16390.07040.0402−0.0262−0.1491−0.1554−0.24140.03400.08280.00360.00950.0146−0.0228−0.0451−0.0031−0.06560.06490.02330.0389−0.0084−0.0120−0.0324−0.01950.01400.0064.The optimal threshold of neurons in the hidden layer is(25)b=−2.42542.6970−2.41431.05300.00960.78091.9744−2.73373.5764.The optimal weight of different neurons between the hidden layer and output layer is(26)ωj=0.3457−0.0424−0.3568−0.1451−1.8627−1.18510.15610.68170.4550.The optimal threshold of neurons in the output layer is(27)b0=1.2744.The first three-order Volterra time-domain kernel can be obtained by substituting the parameters above into equations (15)–(20), and the results are shown in Figures 8–10.Figure 8
The first-order Volterra time-domain kernel calculated by TDANN.Figure 9
The second-order Volterra time-domain kernel calculated by TDANN.Figure 10
The third-order Volterra time-domain kernel calculated by TDANN (m3=0.In order to prove that the identified kernels hold the correct dynamics, predicted responses can be obtained by the first three-order Volterra time-domain kernels, which are shown in Figure11.Figure 11
The predicted response of each assessed kernel.From Figure11, it can be seen that the predicted response using first-order kernel is similar to the system input, which stands for the linear system response, and the predicted response using first two-order kernels begin to appear as nonlinearity components; furthermore, with the order of kernel increasing, the nonlinear components are more and more obvious and also close to the real response.


## 5.3. The Robustness of the Proposed Method

In order to demonstrate the robustness of the proposed method, Gaussian noise is added to the system input; the first three-order Volterra kernels are calculated by TDANN, and then the system output estimated value can be obtained by equation (6), which is shown in Figure 12.Figure 12
The robustness of the proposed method.From Figure12, it can be seen that due to the influence of noise, the estimated output value of the system is in fluctuation state at beginning, and the errors between the estimated value and the true value are relatively large, but after about 2 s, the estimated value begins to approach the true value, and after 4.7 s, the error between the estimated value and the true value is getting smaller and smaller. So, it can be demonstrated that the proposed method has robustness.


## 5.4. The Accuracy Comparison with Traditional Algorithm

In order to verify the accuracy of Volterra kernel calculated by TDANN, the first three-order Volterra time-domain kernels of the nonlinear system are calculated by TDANN, recursive algorithm in reference [32], and variable step size least mean square (VSSLMS) adaptive identification algorithm in reference [33], respectively. Then, three different output results are obtained by equation (6). The accuracy of three methods is shown in Figures 13 and 14.Figure 13
The output comparison of three methods.Figure 14
The errors of three methods.From Figures13 and 14, it can be seen that among three methods, the accuracy of the TDANN method is highest, so each order of Volterra time-domain kernel (as shown in Figures 7–9) calculated by TDANN is closest to the true value of Volterra time-domain kernel. In the VSSLMS identification algorithm, all kinds of parameters have been adjusted to the best, but the identification accuracy is not ideal for the system; two reasons can account for such phenomena. Firstly, the adjustment of parameters in the identification algorithm is based on experience, which is subjective. Secondly, the accuracy of the identification algorithm may be affected by the data type. So, the accuracy of identification algorithm is the lowest. It is worth to notice that in order to further enhance the accuracy, TDANN can easily realize the above three-order Volterra time-domain kernel calculation, while it is difficult for others to calculate the Volterra time-domain kernel above three-order.


## 5.5. The Computation Complexity Comparison with Traditional Algorithm

Both TDANN and recursive algorithm can realize the simultaneous calculation of multiple Volterra kernels, namely, one operation can realize the calculation of multiple Volterra kernels, which can improve the calculation speed and efficiency, while with identification algorithm, one operation can only calculate one Volterra kernel value. In order to demonstrate that TDANN can reduce the computational complexity, the recursive algorithm in reference [32] and the identification algorithm in reference [33] are adopted to randomly calculate several first three-order Volterra kernels of the system. The time consumption of three methods is shown in Table 2.Table 2
The time consumption of different methods.
Samples
Identification algorithm (ms)
Recursive algorithm (ms)
TDANN (ms)
2
113.34
185.69
145.31
4
183.47
6
124.28
(2,3)
129.35
194.16
117.48
(4,7)
175.48
(5,5)
116.72
(1,4,0)
163.89
179.35
104.17
(4,7,0)
142.51
(5,6,0)
176.19It can be seen that the identification algorithm consumed the most time, and TDANN consumed the least time. The time consumption of identification algorithm, recursive algorithm, and TDANN is 1325.25 ms, 559.2 ms, and 366.96 ms, respectively. Compared with identification algorithm and recursive algorithm, the calculation speed of TDANN is increased by 72.31% and 34.38%, respectively. Therefore, the method proposed in this paper reduced the complexity of calculation, which can realize the online calculation and detection of the system.


## 6. Conclusions

To solve the problems of large computation amount and low accuracy in the conventional method to calculate the Volterra time-domain kernel of the nonlinear system, this paper proposed a novel method to intelligently calculate Volterra time-delay kernel by TDANN; by comparing the Volterra kernel model with the TDANN model, the connection of the two models has been found, and the complex Volterra time-domain kernel calculation was transformed into the optimal parameter calculation of neural network; the novel method has been well verified on a nonlinear system.The main achievements can be summarized as follows.Compared with recursive algorithm, the TDANN method has three contributions: Firstly, it can avoid the complex modeling operation of the system; only with input and output of the system, the first three-order Volterra time-domain kernel can be calculated, so it is more adaptable. Secondly, it transforms the complex calculation into the optimal parameter solution of the network, and the Volterra time-domain kernel can be calculated through network training, so it is more intelligent. Thirdly, it can adjust the network structure according to the output of the network to ensure the calculation result is closer to the true value, so it is more adaptive.Compared with the identification method, the TDANN method can not only achieve batch calculation at one time but also reduce the time consumption, so the calculation efficiency is high, which provides the possibility of online calculation.However, the three-order Volterra kernels calculated by TDANN are only parts of the whole Volterra kernel information because the value ofM cannot be too large. How to balance the amount of Volterra kernel information and time consumption is the future work.



---

*Source: 8546963-2020-05-11.xml*