## 3 Novel Broad Echo State Network
### Optimization of Enhancement Layer Based on Unit Increment and Nonstationary Metrics
#### 3.2.2 Parameter Optimization Based on Nonstationary Metrics  
Time series data are often nonstationary in complex systems [41]. The relationships in the nonstationary data are more complicated than that in the stationary time series [42]. The proposed network model is expected to be appropriate for the nonstationary time series. Then the parameters in CMBESN are optimized based on nonstationary time series data analysis. In this paper, the parameters of CMBESN are optimized based on nonstationary metrics. Therefore, the test methods of the nonstationary time series are analyzed first. The classical test methods are Dickey-Fuller [43], augmented Dickey-Fuller (ADF) [44], correlation test [45], etc. The ADF test is a validation method to verify the stationarity of time series in classical econometric theory. The ADF test weakens the influence of the random disturbance term on the overall validation. It does not judge the stability of the time series with the existence of trailing and truncation such as the correlation test. The ADF criterion is the time series' mean and variance. According to the Akaike information criterion (AIC), the degree of nonstationary is determined by judging the probability values (\(p\)), test statistics, 1% critical value, 5% critical value, and 10% critical value. The regularization coefficient \(\lambda\) in the CMBESN is vital for model training. The regularization coefficient is set as the parameter to optimize. This paper proposes an integrated metric consisting of the probability value from the ADF test and the model's errors. The proposed nonstationary metric is as follows. \[S_{i}=f(p)\cdot\frac{1}{n}\sqrt{\sum_{k=1}^{n}(\hat{y}(k)-y(k))^{2}} \tag{19}\]  
\[f(p)=\left|\hat{p}-p\right| \tag{20}\]  
\[\frac{S_{i}}{S_{i+1}}>M \tag{21}\]  
where \(\hat{p}\) and \(p\) denote the ADF probability value of the predicted data and the ADF probability value of the actual data, respectively, \(\hat{y}(k)\) represents the \(k^{\prime}th\) predicted value,\(y(k)\) means the \(k\)_truth_ true value, \(n\) is the number of samples, \(M\) is the set threshold coefficient, and \(S_{i}\) denotes the nonstationary error indicator of the \(i^{\prime}\)_th_ experiment. Combining the above indicators, \(\lambda\) can be calculated by the following formula:  
\[\lambda_{i+1}=\left\{ \begin{array}{l}\alpha\cdot\lambda_{i},L\geq 0\\ \beta\cdot\lambda_{i},L<0\end{array} \right. \tag{22}\]  
\[L=\sum_{k=1}^{n}\dot{y}(k)-y(k) \tag{23}\]  
where \(\lambda_{i}\) denotes the regularization coefficient of the CMBESN model at the \(i\)_trth_ prediction and \(L\) represents the error of the actual data of the prediction series. Formula (19) shows that the proposed metric can reflect the stability difference and the errors between predicted and actual data. Formula (20) is a function of stability difference. The smaller the \(S\), the smaller the prediction error. It can be seen from Formulas (22) and (23) that when the \(L\) results are in the different intervals, the regularized scaling coefficients are adjusted to various degrees. Formula (21) is the termination condition of the optimization process. The optimization flow of the regularization coefficient is shown in Figure 5.  
As described above, the proposed CMBESN is optimized in the enhancement layer. Then it is abbreviated as CMBESN-OE, in which OE means optimizing the enhancement layer. The optimization includes the incremental algorithm of the ESN size and the parameter optimization based on the nonstationary metrics. As a summary of the methods above, the training and test process of the CMBESN is shown in Figure 6. The related algorithms are shown in Tables 2 and 3, and Figure 5.  
Figure 5: Flow chart of optimization based on nonstationary error metrics. ponent in this paper is \(\alpha_{1}=0.2\), \(\alpha_{2}=0.311\), \(\alpha_{3}=0.42\), \(\alpha_{4}=0.51\), \(\alpha_{5}=0.63\), \(\alpha_{6}=0.74\), \(\alpha_{7}=0.85\), \(\alpha_{8}=0.97\), Figure 7 shows the variation of MSO simulation data with eight components.