## 4 Experimental Analysis
### Analysis
#### 4.4.2 Testing Error  
In our work, we selected NRMSE (normalized root mean-squared error) to measure the accuracy of the proposed framework, which is computed by:  
\[NRMSE=\sqrt{\frac{\sum\limits_{i=1}^{K}{{{(y_{i}-d_{i})}^{2}}}}{K\sigma_{d}}} \tag{20}\] \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \#node & 16 & 24 & 32 & 40 & 48 \\  & PSO & & 39,226.026 & & \\  & CNN & & 24,321.529 & & \\  & ADMM & 300.386 & 415.349 & 549.545 & 686.230 & 875.851 \\  & P-PSO & 36.085 & 54.094 & 40.524 & 40.690 & 41.611 \\  & P-PSO (Stage 2) & 23.180 & 41.115 & 27.709 & 27.851 & 28.807 \\ \hline  & \#node & 16 & 24 & 32 & 40 & 48 \\  & PSO & & 30,630.443 & & \\  & CNN & & 27,459.923 & & \\  & ADMM & 411.361 & 528.697 & 668.287 & 836.234 & 1091.287 \\  & P-PSO & 49.181 & 58.554 & 63.622 & 79.493 & 64.106 \\  & P-PSO (Stage 2) & 36.277 & 45.575 & 50.807 & 66.654 & 51.302 \\ \hline  & \#node & 16 & 24 & 32 & 40 & 48 \\  & PSO & & & 26,324.124 & & \\
**Dataset 3** & CNN & & 31,259.718 & & \\  & ADMM & 266.858 & 343.332 & 457.197 & 669.109 & 706.769 \\  & P-PSO & 100.770 & 112.695 & 118.022 & 125.488 & 132.211 \\  & P-PSO (Stage 2) & 87.866 & 99.717 & 105.208 & 112.649 & 119.408 \\ \hline  & \#node & 16 & 24 & 32 & 40 & 48 \\  & PSO & & & 9521.099 & & \\  & CNN & & & 11,865.469 & & \\
**Dataset 4** & ADMM & 2613.941 & 3692.024 & 4837.119 & 5567.593 & 6669.910 \\  & P-PSO & 1230.089 & 1248.736 & 1274.183 & 1259.708 & 1270.916 \\  & P-PSO (Stage 2) & 1207.184 & 1225.758 & 1251.368 & 1236.869 & 1248.113 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparative results of training time. Figure 7: Training time distribution of dataset 1 in lengths of \(1\times 10^{4}\), \(1\times 10^{7}\) and in nodes of 16, 48. In small datasets, less nodes take less time, and the time distribution is stable. Large datasets show the opposite result. where \(K\) is the number of datasets (excluding dropout elements), \(y_{i}\) is the predicted values, \(d_{i}\) is the corresponding true values, \(\sigma_{d}\) is the variance of \(d\). As stated above, we tested each situation 10 times. Figure 8 shows the NRMSE of the former three datasets and the accuracy of the last dataset in box graphs. When the dataset is small, the training error increases with the number of nodes. For large datasets, the training error is relatively stable. That is because different sample sizes may affect the training error. This is obviously the case with the length of \(1\times 10^{4}\). With artificial datasets (Dataset 1 and 2), the training error of our framework is stable, while in real-world datasets (Dataset 3), the training error becomes flickering, which means that features in the real world are more difficult to obtain than artificial data. Figure 8: NRMSE of each dataset in different lengths and nodes. (**a**) NRMSE of dataset 1; (**b**) NRMSE of dataset 2; (**c**) NRMSE of dataset 3; (**d**) Accuracy of dataset 4. Furthermore, the correlation coefficient is used to measure the accuracy of the proposed algorithm on former three datasets. Figure 9 illustrates the results in detail. All three datasets showed that the larger the data size, the higher the correlation coefficient, which indicates a better result. It also confirms the above conclusions from the NRMSE. We compared the testing error between the traditional PSO, ADMM, CNN and P-PSO-based algorithms in this paper, with a typical length of data size of (\(1\times 10^{5}\)); the result is shown in Figure 10.  
As shown in Figure 10, for artificial datasets (datasets 1 and 2), the P-PSO-based optimization algorithm is clearly outperforming the other three algorithms, and it even performs well in the real-world dataset (dataset 3). However, when images (high-dimension data) are treated, CNN shows an excellent score (1.2%). The comparison between the four algorithms indicates that CNN is good at two-dimensional image recognition, the P-PSO-based ESN algorithm has a better ability to handle time-series data, and ADMM is slightly inferior; the traditional PSO is the most unsteady algorithm.