## 3 Results
### Testing the Model Forecasts  
Several tests were conducted to evaluate the forecast during the time interval \([T_{e},\,T_{e}+\Delta T]\) by comparing the simulated results with the observed seismicity. To check the performance of forecasts for the number of aftershocks and magnitude distribution, the N and M-tests were performed, respectively. The details of the implementation of the tests can be found in Shcherbakov [4]. For the three aftershock sequences the number of forecasted aftershocks in the forecasting time interval \(\Delta T=7\) days and using the following target time intervals \(T_{e}=[1,\,2,\,3,\,4,\,5,\,6,\,7,\,10,\,14,\,21,\,30]\) days are given in Figure 10. For comparison, in the same figure the observed number of earthquakes are shown as blue circles for each prediction time interval \(\Delta T\). In addition, for the constant target time interval \([T_{s},\,T_{e}]=[0.06,2]\) days and varying forecasting time intervals \(\Delta T=[1,\,2,\,5,\,7,\,10,\,14]\) days the number of forecasted and observed earthquakes are shown in Figure 10. To investigate the effect of the magnitude cutoff, we also performed the same analysis for earthquakes above magnitude 3.5. This is reported in Figure 10.  
Figure 10: _Cont._  
In addition, M-test was performed to assess the consistency of the distribution of the magnitudes of the forecasted events. The results of the performance of the OU law and ETAS model are reported by computing the quantile score, \(\kappa\)[4,19]. \(\kappa\) is defined as the proportion of the forecasted magnitudes compared to the observed magnitudes in each magnitude bin. The obtained quantile scores for the constant forecasting time interval \(\Delta T=7\) days are given in Figure 11. In addition, in Figure A11 the outcomes for the M-test for the constant target time interval \([T_{s},\ T_{e}]=[0.06,\ 2]\) days are plotted for the three aftershock sequences. Figure 11: _Cont._  
Figure 10: The number of forecasted and observed aftershocks in the forecasting time interval \(\Delta T=7\) days for the three sequences: (**a**) the 3 November 2002, Mw 7.9 Denali sequence for \(m\geq 3.0\); (**b**) the 23 January 2018, Mw 7.9 Kodiak sequence for \(m\geq 3.2\); (**c**) the 30 November 2018, Mw 7.1 Anchorage sequence for \(m\geq 2.8\). The red triangles show the average number of forecasted earthquakes using the ETAS model and the black squares illustrate the average number of forecasted earthquakes using OU law. The shading bands represent 95% confidence intervals. The blue circles represent the observed number of earthquakes in each forecasting time interval. In order to evaluate and compare the models, the R-test and T-test were applied for both cases by considering the ETAS model versus the OU law. In the R-test the quantile score, \(\alpha\), was calculated. \(\alpha\) is the proportion of the simulated likelihood ratios, over the observed likelihood ratios [17]. The values of \(\alpha\) that are greater than a specific level of significance support the model that was chosen as a base model, in this case it is the ETAS model. The obtained result for the \(\alpha\) from the OU law versus the ETAS model is shown in Figure 12. Furthermore, in Figure 13, the ratio of the likelihood score of the ETAS model and the OU law over the number of the observed events in the forecasting time interval is given. The T-test is used to assess whether the sample information gain is statistically different from zero. This is used to select the preferred model [49]. Lastly, the Bayesian \(p\)-value of the BPD analysis using either the ETAS model or the OU law are illustrated for both cases in Figure 14.  
Figure 11: The obtained quantile scores from the M-test for the constant forecasting time interval \(\Delta T=7\) days for the three sequences: (**a**) the 3 November 2002, Mw 7.9 Denali for \(m\geq 3.0\); (**b**) the 23 January 2018, Mw 7.9 Kodiak for \(m\geq 3.2\); (**c**) the 30 November 2018, Mw 7.1 Anchorage for \(m\geq 2.8\).