## 2 Materials and Methods
### Confusion Matrix Calculation  
Finally, the ionospheric scintillation variation data and the occurrence of earthquakes, are analyzed to assess the correlation of their mutual occurrence, and the confusion matrix is computed. The overall process is shown in the diagram in Figure 6.  
A confusion matrix (CM) is the usual way to study the effectiveness of an algorithm to detect an event. It is a 2 \(\times\) 2 matrix classifying the events according to two binary classes: the predicted ionospheric scintillation signature, and the actual occurrence of the earthquake. This classification results in four possible cases, as depicted in Figure 7.  
Figure 5: Histogram of earthquakes per magnitude during the period studied for all places (blue) and the ones happening on land (brown). Figure 6: Diagram showing the complete data processing, from raw data sources to the CM inputs. Figure 7: Confusion matrix diagram. In particular, after the data pre-processing described in previous sections, the creation of the confusion matrix is described below, repeating this classification for each day \(d\) and each pixel \((x,y)\) within the \(S_{4}\) variations map:  
* Predicted condition, \(S_{4}\) positive increment (\(\Delta S_{4}\)):
* Yes (PP: Predicted Positive): If the \(\Delta S_{4}\) value in the current pixel is larger than a certain threshold \(s4\_th\). * No (PN: Predicted Negative): If the \(\Delta S_{4}\) value in the current pixel is smaller than the threshold. * Actual event: earthquake:
* Yes (P: Positive event): if at least one earthquake with a magnitude within the magnitude interval defined (\(eq\_mag\)) has its epicenter within a certain distance (_radius_) from the center of the pixel \((x,y)\), and it occurs within a certain time window \([d+d\_margin]\) (in days) from the studied day. Please note that this way, a positive \(d\_margin\) means that the earthquake takes place after the ionospheric anomaly. * No (N: Negative event): if any earthquake matches the previous criteria. The matrix accounts for the total number of events in each of the four categories. As planned, each event studied fit in one, and only one, category. The maximum number of events computed in a CM will be \(N\_days\times N\_lats\times N\_lons\) events, where \(N\_days\) is the number of days under study and \(N\_lats\times N\_lons\) is the number of pixels in the map. In our case, using \(1^{\circ}\times 1^{\circ}\) pixels, there are \(80\times 360=28,800\) points. As explained in Section 2.2, in the S\({}_{4}\) variations maps there could be pixels with no data, either because they are completely located on land, and never fit a specular reflection point or because the LT filter applied removed all the days in the current small window used. This means that the total number of pixels computed (TOT) will be less than the total on the map, in particular, every day, a constant number referring to the land coverage will be subtracted, and then another quantity corresponding to the temporal evolution shown in Figure 2.  
Several confusion matrixes are constructed after setting the different parameters mentioned before, which are detailed in Table 2:  
The \(d\_margin\) parameter is used to study whether there is a preferred time of appearance of the ionospheric anomalies before or after the earthquakes. Please note that when a CM is labeled as \(d\_margin=+6\), it means that for each day \(D\) in which the anomaly is computed, earthquakes are searched within the days \((D+3,D+6)\) days, i.e., the anomalous change in \(S_{4}\) is a precursor of the earthquake. For example, \(d\_margin=-3\) would mean from \((D-3,D+0)\) days.