# Evolutionary Optimization of Spiking Neural P Systems for Remaining Useful Life Prediction
## 5 Numerical Results
This measurement helps compare the computational simplicity of each method intuitively. Considering the stochasticity of the evolutionary search conducted by NEAT, to improve the reliability of the results, we execute 10 independent runs, each one with a different random seed. At the end of each run, we calculate the RMSE on the test set obtained by the best solution found in that run (note that during the evolutionary process, each solution is instead evaluated on the training set). We consider the mean of these 10 RMSE values as the final performance of the obtained SN P systems. For illustration purposes, Figures 3 and 4 show the fitness trend (i.e., the training RMSE trend), in terms of average (solid line) \(\pm\) std. dev. (shaded area) across 10 independent runs, for the best solutions found across generations with two selected NEAT configurations on the two datasets, respectively. From the trends, we can conclude that the algorithm quickly converges in about one-third of the budget and then achieves minor improvements in the remaining part of the evolutionary process. Furthermore, we can observe that the method is quite robust across runs, since the std. dev. of the fitness trend is fairly small. Figure 3: Fitness across generations (mean \(\pm\) std. dev. across 10 independent runs) for SNPS (5) on CMAPSS (FD001). Concerning CMAPSS, the comparative results on the FD001 of all the considered methods are presented in Table 7. SNPS (1) to (6) indicate the best SN P systems obtained with each of the six NEAT configurations reported in Table 6. In terms of the test RMSE, all the six results are much better than the MLP, since we can achieve lower RMSE values with a smaller number of trainable parameters. In addition, we observe that the test RMSE of the SVM-based model taken from [38] is placed between the result of the MLP and our best results. Furthermore, the test results of SNPS (4) and (6) are fairly comparable with those obtained by the CNN and the LSTM in terms of test RMSE, but our proposed method achieves these results by using a considerably smaller number of trainable parameters (1-2 orders of magnitude lower). In particular, we should note that the solution that gives the best RMSE, SNPS (5), contains only 232 parameters, while the most complex method in the table, the LSTM, has 14,681 parameters. The architecture of this SNPS is depicted in Figure 5. It should be noted that this system was simplified by removing the nodes and edges that did not have an impact on the output. Considering the \(s\)-score, the values obtained by our proposed method are better than the \(s\)-score of the MLP. Compared to the CNN, SNPS (5) gives a better \(s\)-score, although its RMSE is slightly worse. This indicates that the proposed method not only speeds up the RUL predictions with comparable accuracy, but also it can be robust to the risk of optimistic RUL predictions. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **Test RMSE** & **\(s\)-Score \(\times\) (10\({}^{3}\))** & \
\begin{tabular}{c} **Trainable** \\ **Parameters** \\ \end{tabular} &
\begin{tabular}{c} **Test Execution** \\ **Time (ms)** \\ \end{tabular} \\ \hline MLP [13] & 37.36 \(\pm\) 0.00 & 6.45 & 801 & 93 \\ CNN [13] & 18.45 \(\pm\) 0.00 & 1.29 & 6815 & 151 \\ LSTM [14] & **16.14 \(\pm\) 0.00** & **0.34** & 14.681 & 968 \\ SVM [38] & 29.82 \(\pm\) 0.00 & - & - & - \\ SNPS (1) & 29.27 \(\pm\) 3.76 & 3.28 & 263 & 25 \\ SNPS (2) & 31.55 \(\pm\) 3.61 & 8.83 & 365 & 27 \\ SNPS (3) & 29.28 \(\pm\) 2.99 & 4.36 & **60** & **7** \\ SNPS (4) & 20.90 \(\pm\) 1.52 & 0.81 & 334 & 28 \\ SNPS (5) & 20.32 \(\pm\) 1.54 & 0.54 & 232 & 25 \\ SNPS (6) & 20.79 \(\pm\) 1.58 & 1.57 & 285 & 26 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Summary of the comparative analysis based on the test results on CMAPSS (FD001). The symbol “-” indicates not available data. For our SNPS methods, we report the RMSE in terms of mean \(\pm\) std. dev. across 10 independent runs. For the remaining methods, we report the results from the original papers (std. dev.