# Predicting the Magnitude of Earthquakes Using Grammatical Evolution
## 3 Experimental Results
LSTM and MLP-RPROP are the best non-evolutionary models (average error \(\sim\)0.24), while MLP-BFGS shows significant instability (average 0.802), with particularly high errors at short distances. In Figure 9, the grammatical models maintain their superiority with an average error of 0.176-0.188, with NNC standing out. LSTM continues to be the top non-evolutionary model (average 0.24), with remarkable stability, while MLP-BFGS improves compared to 2004 (average 0.548), but retains significant variation, especially at short distances. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline
**YEAR** & \(\mathbf{D_{c}}\) & **LSTM** & **SVM** & **MLP (BP)** & **MLP (RPROP)** & **MLP (BFGS)** & **RULE** & **NNC** & **FC** \\ \hline
2004 & 10 & 0.25 & 0.29 & 0.44 & 0.24 & 0.82 & 0.16 & 0.16 & 0.17 \\ \hline
2004 & 25 & 0.23 & 0.29 & 0.42 & 0.24 & 0.98 & 0.17 & 0.17 & 0.1  
In Figure 10, the grammatical models are shown to achieve their best performance, with FC (average 0.166) and NNC (0.17) demonstrating exceptional stability. LSTM remains the most reliable non-evolutionary model (average 0.224), while MLP-BFGS shows extreme variation (from 0.21 to 0.85), despite its reduced average error (0.566). Figure 8: Error per point for each method used on the 2004 dataset, and different values of the critical parameter \(D_{c}\). Figure 10: Error per point for the machine learning methods used on the 2012 dataset, and a series of values for the critical parameter \(D_{c}\). Figure 9: Error per point for each used method used on the 2010 dataset, and a variety of values of the critical parameter \(D_{c}\). According to the results of the Friedman test shown in Figure 11, where the overall difference between the models is extremely statistically significant (\(p\) = 4.51 \(\times\) 10\({}^{-18}\) < 0.0001, critical value = 4.2863,critical difference = 3.834), the following key stratifications are observed: The models with Grammatical Evolution (RULE, NNC, FC) show homogeneous performance (\(p\) = ns among them), but are significantly better (\(p\) = ****) than MLP(BP) and MLP(BFGS). They also significantly outperform SVM (\(p\) = **** for NNC/FC, \(p\) = **** for RULE). LSTM (without Grammar-Based Evolution) does not differ significantly either from the models with Grammar-Based Evolution (\(p\) = ns) or from SVM, MLP(BP), and MLP(RPROP) (\(p\) = ns), with the only significant difference being against MLP(BFGS) (\(p\) = **). MLP(RPROP) (without Grammar-Based Evolution) does not differ from RULE (\(p\) = ns), but shows significantly higher error than NNC/FC (\(p\) = *) and MLP(BFGS) (\(p\) = *). MLP(BFGS) (without Grammar-Based Evolution) is the least reliable model, with significantly worse performance (\(p\) = ****) compared to all models with Grammar-Based Evolution, and significantly worse performance than LSTM (\(p\) = **) and MLP(RPROP) (\(p\) = *). In Figures 12 and 13, the prediction error analysis results for different machine learning models are presented. We observe significant differences between the two model categories. The analysis reveals that models without Grammatical Evolution exhibit greater variability in prediction errors. Notably, one of these models shows particularly high errors at certain distances, indicating a lack of stability. In contrast, the most effective model in this category maintains relatively low errors across all distances. The methods with Grammatical Evolution demonstrate remarkable stability and lower errors. The three models in this category show similar and consistent performance, with minimal variations across different distances and years. The effect of distance is clearly visible in models without Grammatical Evolution, where errors vary significantly. Conversely, in models with Grammatical Evolution, distance does not appear to significantly affect prediction accuracy. Temporally, while models without Grammatical Evolution show some improvement over time, models with Grammatical Evolution maintain stable performance throughout the study period.