## 4 Discussion
### Validation of the Results by Confusion Matrix Performance Evaluation and ROC Curves  
The results have been evaluated by the so-called \"confusion matrix\", which is a table of two rows \(\times\) two columns used to assess the performances of a prediction model, crossing the presence (or not) of an ionospheric anomaly followed (or not) by an earthquake in the given time within the pre-chosen distance and time. Here, we considered the same anomalies presented in the results, but we evaluated a cell that has 6\({}^{\circ}\) geomagnetic latitude for 6\({}^{\circ}\) geographic longitude and 2.4 days (for investigations previously conducted 90 days before the earthquake) or 20 days (for investigations conducted 500 days before the earthquake). The evaluation of the cell has been selected among the classic four cases:  
1. True positive (_TP_): in the cell, there is at least an anomaly, and an earthquake follows within the next 90 days
2. True negative (_TN_): in the cell, there are no anomalies, and no earthquakes follow in the next 90 days in the same cell. 3. False positive (_FP_): in the cell, there is one or more anomalies, and no earthquakes follow in the next 90 days in the same cell
4. False negative (_FN_): in the cell, there are no anomalies, and an earthquake follows in the next 90 days. From the evaluation of the number of true and false positives and negatives, the accuracy \"_Acc_\" has been computed as the total number of good cases over the total cases: \[Acc=\frac{TP+TN}{TP+TN+FP+FN}\] (5)  
Better performance with respect to a random prediction is achieved when \(Acc>50\%\). We also calculated the Hit Rate (_HR_) and False Positive Rate (_FR_); the former expresses the percentage of predicted events, while the latter is the rate of false alarms:  
\[HR=\frac{TP}{TP+FN}\quad FR=\frac{FP}{FP+TN} \tag{6}\]  
An ideal prediction system would give the limit case with \(HR=100\%\) (all events are predicted) and \(FR=0\%\) (no false alarms), while a real system is better as close as possible to such limit case, and, typically, the earthquake prediction systems are not suitable for use with the population when the false positive rate is too high. Finally, the Alarmed Time-Space \"_AT_\" (also called the alarm rate in the literature [39]) has been estimated:  
\[AT=\frac{TP+FP}{TP+TN+FP+FN} \tag{7}\]  
This quantity is important to check that a good prediction is not just achieved regarding an analyzed space-time that is too large or even an entire space-time. For example, the affirmation that \"_in the future, in the World, an M5.5+ earthquake will occur_\" is surely true, but it is not a \"prediction\", because it does not predict the exact time, and the space domain is too large (the whole world). Given the same accuracy, a prediction is preferable with a lower _AT_. The alarmed time can also be compared with the number of positive cases (_TP + FN_) with respect to the total cases, as, even an ideal (so, perfect) prediction needs to alarm at least all of the space-time that any event occurred. Table 1 reports the evaluation of performances in terms of confusion matrices for the _Swarm_ magnetic field and electron density data considering 90 days and 500 days before the M5.5+ earthquakes. These tables reflect the analyses shown in Figures 1, 3, 4, S2, S4 and S5, even if we consider only the time before the earthquakes, as the confusion matrix is a tool to assess the prediction capability of a system, so only the ionospheric anomalies that preceded the earthquake can be \"positively\" evaluated. It is possible to note that all the analyses show a high accuracy, from 72% until 95%. Contrariwise, the hit rates tend to be rather low (i.e., from ~1% to ~19%), as several earthquakes are not preceded by anomalies in the considered interval of time, and, overall, several earthquakes did not show anomalies continuously in the 90 (or 500) days before their occurrence.