# Towards a Real-Time Description of the Ionosphere:
## 4 Methodologies of Analysis  
### Statistical Metrics Adopted in the Validation Process  
To evaluate the IRI and IRTAM models' performances, different statistical metrics were calculated between measured and modeled _fo_F2 and _hm_F2 values. Specifically, the mean of the residuals (_Res. Mean_) between measured and modeled (by IRI and IRTAM) values, the root mean square error (_RMSE_), the normalized root mean square error (_NRMSE_), and the Pearson correlation coefficient (_R_) are the considered statistical metrics:  
\[Res.Mean =\frac{\sum_{i=1}^{N}(xx\text{F2}_{\text{measured},i}-xx\text{F2}_{ \text{model},i})}{N}, \tag{1}\]  
\[RMSE=\sqrt{\frac{\sum_{i=1}^{N}(xx\text{F2}_{\text{measured},i}-xx\text{F2}_{ \text{model},i})^{2}}{N}}, \tag{2}\]  
\[NRMSE\ \left[\%\right]=\frac{\text{RMSE}(xx\text{F2}_{\text{measured},xx \text{F2}_{\text{model}}})}{xx\text{F2}_{\text{measured}}}\cdot 100, \tag{3}\]  
\[R=\frac{\text{cov}(xx\text{F2}_{\text{measured},xx\text{F2}_{\text{model}}}) }{\sigma_{xx\text{F2}_{\text{measured}}}\sigma_{xx\text{F2}_{\text{model}}}} \in[-1,1], \tag{4}\]  
where \(xx=f\)o when considering _fo_F2 and \(xx=hm\) when considering _hm_F2. The subscript \"modeled\" refers to values obtained through IRI or IRTAM, while the subscript \"measured\" refers to values observed by either ionosondes or COSMIC satellites; the index \(i\) runs on the \(N\) values of the time series. \(\overline{xx\text{F2}_{\text{measured}}}\) is the arithmetic mean of \(xx\text{F2}_{\text{measured}}\), cov is the covariance between modeled and measured values, and \(\sigma\) the corresponding variance. The unit of measurement of both _Res. Mean_ and \(RMSE\) can be either MHz or km, for _fo_F2 and _hm_F2, respectively. It is important to point out that _Res. Mean_ highlights the accuracy of modeled values, while _RMSE_ and _NRMSE_ are indicators of the corresponding precision.  
Concerning the validation with the COSMIC dataset also the root mean square percentage error (_RMSPE_) was considered:  
\[RMSPE\ \left[\%\right]=\sqrt{\frac{\sum_{i=1}^{N}\left(\frac{xx\text{F2}_{ \text{measured},i}-xx\text{F2}_{\text{model},i}}{xx\text{F2}_{\text{measured},i}} \cdot 100\right)^{2}}{N}}. \tag{5}\]  
Differently from _NRMSE_, when calculating _RMSPE_, the normalization of the residuals between measured and modeled values is performed point-by-point before calculating the root mean square value. The use of _RMSPE_ is more suitable than that of _NRMSE_ for datasets with very different absolute values for different diurnal, seasonal, or solar activity conditions. In principle, this is true for both datasets, the COSMIC and the ionosonde ones. Nevertheless, the COSMIC dataset is global, that is it varies with the geographical coordinates, while the location of ionosondes is fixed. This is why _RMSPE_ is suited especially for the COSMIC dataset.  
In order to evaluate the deviations between models and observations, the residuals deviation ratio parameter \(R_{\text{cw}}\)[18] was also computed:  
\[R_{\text{cw}}=\left|\frac{\varepsilon_{\text{c}}}{\varepsilon_{\text{w}}} \right|=\left|\frac{xxx\text{F2}_{\text{IRI}}-xx\text{F2}_{\text{measured}}}{ xx\text{F2}_{\text{IRI}}-xx\text{F2}_{\text{measured}}}\right|, \tag{6}\]  
where \(\varepsilon_{\text{c}}\) represents the residuals between \"climatological\" (i.e., IRI modeled) and measured values, and \(\varepsilon_{\text{w}}\) are the corresponding residuals between \"weather\" (i.e., IRTAM modeled) and measured values.  
The statistical parameter \(R_{\text{cw}}\) allows us to directly compare IRI and IRTAM models and easily evaluate the corresponding performances. In fact, by definition, \(R_{\text{cw}}=1\) when both models are on par in terms of their accuracy, while \(R_{\text{cw}}>1\) when IRTAM is more accurate than IRI and \(R_{\text{cw}}<1\) when IRI is more accurate than IRTAM.