# A Kernel Least Mean Square Algorithm Based on Randomized Feature Networks
## 5 Simulations and Results
### Lorenz Time Series Prediction  
The problem setting for time series prediction is based on the previous five inputs as the input vector \([x(n-5),...,x(n-1)]^{T}\) to predict the current points \(x(n)\). A chaotic time series is a highly nonstationary time series in the real world [28; 29]. The Lorenz model can be described as the three ordinary differential equations:  
\[ \begin{split}\frac{dx}{dt}&=a(y(t)-x(t))\\ \frac{dy}{dt}&=bx(n)-y(t)-x(t)z(t)\\ \frac{dz}{dt}&=x(t)y(t)-cz(t)\end{split}  \tag{38}\]  
where \(a\), \(b\), and \(c\) are set as 10, 8/3, and 28, respectively [30]. The step size in the fourth-order Runge-Kutta method is 0.01. The time delay of the prediction is chosen as 1. The length of generated training samples is 4200, among which the first 200 data are discarded to wash out the initial transient. The left 4000 samples are used as the training set. An extra 200 samples are generated as the testing data to measure the prediction performance. The zero-mean Gaussian white noise with the noise level of 20 dB signal-to-noise ratio (SNR) is added to the time series. For all algorithms compared in this simulation, the step size is set to 0.1. Figure 1 shows the Lorenz time series. To evaluate the impact of the dimension parameter of KLMS-RFN on the steady state error, the kernel parameter \(\sigma\) of KLMS-RFN is set to 0.707. After fixing the kernel parameter and the step size, the steady state performance of KLMS-RFN is only affected by the dimension \(M\). The value of parameter \(M\) ranges from 10 to 100, with an interval of 10. The smaller MSE equates to better performance in the comparison of different algorithms. As we can see from Figure 2, the final testing MSE of KLMS-RFN becomes better as the dimension increases. However, the performance will not be significantly improved as the dimension increases to a certain extent because the more features that are used to predict the time series, the better the predicting accuracy will be. When the number of dimensions increases to a certain extent, the number of redundant sample features will increase, leading to a slower improvement in accuracy. Figure 1: The Lorenz time series to be processed. To evaluate the influence of kernel parameter \(\sigma\) on the steady state error of the KLMS-RFN algorithm, the dimension is chosen as 50, and the value of the kernel parameter \(\sigma\) ranges from 0.4 to 1.6. Figure 3 shows that the final MSE varies from \(-11.1\) dB to \(-15.8\) dB and finally increases to \(-14\) dB under different \(\sigma\). The optimum value of the kernel parameter is 1. Both small and large kernel parameters will lead to accuracy degradation. To compare the performance of the proposed algorithm with the QKLMS and QKLMS-FB algorithm, the dimension number of the KLMS-RFN is fixed at 50. The kernel parameter of the QKLMS, QKLMS-FB, and KLMS-RFN is chosen as \(\sigma=1\). The parameters \(\gamma\) and \(\beta\) of the QKLMS-FB are set to 0.4 and 0.96, respectively. The \(\gamma\) of the QKLMS is set to be the same as for the QKLMS-FB. The budget of the QKLMS-FB algorithm is set to 25. The testing MSE curves and the network size growth curves averaged over 100 Monte Carlo simulations are shown in Figures 4 and 5. The final network size of the QKLMS-FB achieves the preset budget. The final network size of the QKLMS is larger than that of the the QKLMS-FB, which means more memories and computations are required by the QKLMS when calculating the kernel matrix. The final MSE test values for the QKLMS, QKLMS-FB, and KLMS-RFN are \(-15.37\) dB, \(-12.46\) dB, and \(-15.37\) dB, respectively, and are calculated by averaging the last 200 samples. The smaller MSE equates to better steady state performance in the comparison of different algorithms. The proposed algorithm shows the similar performance in accuracy to the QKLMS-FB when the dimension is chosen to be 50. Meanwhile, the accumul  
Figure 3: The final testing MSE curve of KLMS-RFN algorithm with the kernel parameter \(\sigma\) varying from 0.4 to 1.6.