## 4 Discussion
In particular, the values of the DOR are shown in Figure 16. The DOR, given by Equation (7), is an indicator of the diagnostic performance that takes into account the whole parameters in the CM and it ranges from 0 to infinity, being larger than one for positive correlations, and higher for better performance. In the plots, it is shown that it goes from 1 to 3 or 4 (depending on the case) as s4_th increases. Figure 16a, for the magnitudes from 4 to 4.9 we can see that all the curves for different \(d\_margin\) values are very similar, and they separarte for the largest magnitude intervals, as in Figure 16c, where the largest DOR is almost 5 for a \(d\_margin=+6\) and s4_th = 0.024. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**ROC-AUC 50 km** & \multicolumn{4}{c}{\(d\_margin\) (Days)} \\ \hline
**Magnitudes** & **[\(-\)6,\(-\)3]** & **[\(-\)3,0]** & **[0,\(+\)3]** & **[\(+\)3,\(+\)6]** \\ \hline
4–4.9 & 0.5117 & 0.5180 & 0.5232 & 0.5193 \\
5–5.9 & 0.5297 & 0.5202 & 0.5201 & 0.5141 \\ \(\geq 6\) & 0.4670 & 0.4753 & 0.4881 & 0.5199 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: ROC-AUC values for \(radius=50\) km. \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**ROC-AUC 100 km** & \multicolumn{4}{c}{\(d\_margin\) (Days)} \\ \hline
**Magnitudes** & **[\(-\)6,\(-\)3]** & **[\(-\)3,0]** & **[0,\(+\)3]** & **[\(+\)3,\(+\)6]** \\ \hline
4–4.9 & 0.5204 & 0.5249 & 0.5256 & 0.5242 \\
5–5.9 & 0.5223 & 0.5250 & 0.5278 & 0.5168 \\ \(\geq 6\) & 0.5148 & 0.5487 & 0.5766 & 0.5635 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: ROC-AUC values for \(radius=100\) km. Another metrics computed is the accuracy (ACC) using Equation (4), which is shown in Figure 17 for the 100 km radius case. The problem with this indicator for very unbalanced classifications such as this is that they will give unrealistically high values because they are only accounting for the TP and TN counts with respect to the total TOT. It is observed in the plot that it goes from 50% accuracy to almost 100%, and all the curves are overlapping in the one shown here, for all \(d\_margin\), and all magnitude intervals. In this case, as the number of Positives (P) and Negatives (N), i.e., earthquakes vs. no-earthquakes, is very unbalanced, the accuracy is not a good indicator of correlation. To complete the analysis, curves for the F\({}_{1}\) score and Matthews Correlation Coefficient (MCC) are shown in Figures 18 and 19, respectively. F\({}_{1}\) score is an improvement of the ACC parameter, but still, it is less indicative than the DOR or the MCC for this case. Remember from Equation (5), that the numerator is only accounting for the true positive (TP) cases, which for this study are decreasing a lot for large magnitudes intervals, which explains the low values in the plot in Figure 18c. Figure 16: DOR values for two magnitude intervals using \(radius=100\) km with respect to the \(\Delta S_{4}\) threshold: \(s4\_th\). Figure 17: ACC values for two magnitude intervals using \(radius=100\) km with respect to the \(\Delta S_{4}\) threshold: \(s4\_th\). Finally, the MCC is shown in Figure 19a-c and it indicates positive values for all magnitude intervals. MCC ranges from \(-1\) to \(+1\), being zero when there is no correlation. In the same way as other parameters studied here, MCC is indicating positive values although they are small. As the F\({}_{1}\) score, the MCC shows smaller values for the largest earthquake magnitudes, the height of the peak is decreasing from Figure19a to Figure19c. An interesting conclusion that can be extracted from the two most significative statistical parameters (DOR and MCC), is the value of the ionospheric scintillation threshold (\(s4\_th\)) parameter that is giving the best performance when correlating earthquakes with \(S_{4}\) positive variations. As observed in Figure 16a,b for small magnitude intervals (from 4 to 6), using the DOR metrics, \(s4\_th\) is larger than 0.03 (the largest value swept), but it is close to 0.02 for magnitudes larger than 6, in Figure 16c.