# COVID-19: A Comparison of Time Series Methods to Forecast Percentage of Active Cases per Population
## 5 Experiments and Results
### Model Performance  
Performance results of each model for each country are presented in Table 1, indicating that, in terms of RMSE, there is no one-size-fits-all approach for predicting the percentage of active cases with respect to the total population for different countries.  
Figure 4: Percentage of active cases with respect to the total population per country. For each country, moment zero is the first confirmed case in that country.  
Moreover, in terms of statistical testing, the Friedman non-parametric multiple groups test [63] with a significance level of \(\alpha=0.02\) was applied to produce the corresponding ranking between the algorithms, as shown in Table 2  
Overall, as demonstrated both by RMSE-measured performance (Table 1) and by the statistical ranking (Table 2), statistical approaches such as ARIMA, TBAT and HWAAS outperform the deep learning methods such as N-BEATS and DeepAR (GluonTS), which is probably a result of the lack of high volumes of data that deep-learning algorithms need to thrive.  
It is also worth noting that Facebook's Prophet did not achieve superior performance in any of the countries, which did not come as a big surprise, given that the method was fundamentally developed to deal with business problems.  
That said, firstly TBAT and secondly ARIMA seem to be the best performing methods overall, providing, in terms of RMSE, the best predictions in a combined seven countries, while also achieving second best results in another two. More specifically, ARIMA provided superior performance in terms of RMSE in the USA, Italy and Russia (Figure 5), while TBAT achieved the best results in Spain, the UK (Figure 6), Germany and Iran.  
Delving deeper into TBAT, in nine out of ten cases, it made it into the top two, with the exception of Brazil where it achieved third-best performance, behind Gluonts and ARIMA.  
Furthermore, in terms of statistical testing, when [PERSON]'s post-hoc statistical analysis [64] was applied on top of [PERSON]'s test, it proved that, by using significance level of \(\alpha=0.02\), TBAT is significantly better than Prophet, DeepAR (Gluonts) and N-BEATS, as illustrated in Table 3.  
Further into statistical time series methods, regarding ARIMA and HWAAS, while the former performed better overall than the latter, both achieved top-three performance in seven out of ten cases with the Holt-Winters additive model producing the best results in Turkey.  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline  & **US** & **Spain** & **Italy** & **UK** & **France** \\ \hline ARIMA & **0.007421** & 0.080094 & **0.005628** & 0.005484 & 0.060824 \\ Prophet & 0.013877 & 0.065433 & 0.019217 & 0.007634 & 0.044482 \\ HWAAS & 0.172957 & 0.031497 & 0.006616 & 0.004366 & 0.011007 \\ NBEATS & 0.036958 & 0.050492 & 0.008645 & 0.037623 & **0.004220** \\ Gluonts & 0.044805 & 0.108842 & 0.043551 & 0.046134 & 0.010549 \\ TBAT & 0.009873 & **0.029295** & 0.005810 & **0.004310** & 0.007003 \\ \hline  & **Germany** & **Russia** & **Turkey** & **Brazil** & **Iran** \\ \hline ARIMA & 0.006431 & **0.001536** & 0.004442 & 0.004194 & 0.002628 \\ Prophet & 0.037139 & 0.014681 & 0.044595 & 0.009279 & 0.016281 \\ HWAAS & 0.004586 & 0.002295 & **0.000887** & 0.005717 & 0.001046 \\ NBEATS & 0.013192 & 0.027078 & 0.018265 & 0.010870 & 0.003745 \\ Gluonts & 0.057523 & 0.034479 & 0.093839 & **0.002836** & 0.002277 \\ TBAT & **0.003389** & 0.002193 & 0.001946 & 0.005621 & **0.000425** \\ \hline \hline \end{tabular}
\end{table}
Table 1: Model performance in terms of RMSE for ten countries with the most confirmed COVID-19 cases as of 4 May 2020.  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Rank** & **Algorithm** \\ \hline
1.70000 & TBAT \\
2.90000 & ARIMA \\
2.90000 & HWAAS \\
4.10000 & NBEATS \\
4.60000 & Prophet \\
4.80000 & Gluonts \\ \hline \hline \end{tabular}
\end{table}
Table 2: Friedman statistical test ranking (significance level of \(\alpha=0.02\)).