## 6 Precision
### CNM Precision Validity Score
Compared to that of method (1) with the Standard Normal Interval, (2) takes into account the correction of the standardized estimator of the sample mean of normally distributed data with a small sample size. (3), (4), and (5) are based on the bootstrapping method. Bootstrapping is generally useful for estimating the distribution of a statistic (e.g., mean, variance) without using normal theory. Bootstrapping [31; 32] accounts for the exact distribution of the underlying measurement parameter and falls under the broader class of resampling methods. This is particularly necessary if arbitrary distributed measured values are obtained from the CNM. Bootstrapping estimates the properties of a distribution by measuring those properties from a sample. Bootstrapping and jackknife methods were proven to be powerful tools for approximating the sample distribution and variance. The bootstrap values can be determined as follows: (1) \(B\) random bootstrap samples are generated, (2) a parameter estimate is calculated from each bootstrap sample, (3) all \(B\) bootstrap parameter estimates are ordered from lowest to highest when calculating the Percentile Confidence Interval, and (4) the CI is constructed accordingly. BCa confidence intervals adjust for skewness in the bootstrap distribution, but since CNMs in particular often have to evaluate huge amounts of data, this can be computationally intensive for many users. According to our preliminary investigations and the given the size of our data set, the results on our server, namely a Super Micro server with 96 CPU cores and 1008 GB RAM, were practically incalculable. For individual regions with more than 100k measurements, the calculation took more than a day with results without any significant difference compared to the other bootstrapping methods for the downlink throughput in France. The pseudocode for calculating the CNM Precision Validity Score can be found in Algorithm 1. One input parameter is the type of calculation with _type_ = {_Normal_! _Stud-t_! _Bootstrap_}. Basically, this algorithm can be used for all types. If _Stud-t_ is used, however, the result is an estimate, as described in Section 6.1. Therefore, a further algorithm is given in Algorithm 2, which allows the exact calculation on the basis of an iterative method. Here, a _repeat_..._until_ loop is used to calculate the exact value of the quantile of the Student's! \(t\) distribution after every increase in \(\hat{m}\), since it depends on it (cf. degrees of freedom). Note that a further extension for the practical implementation of the algorithms would be a binary search instead of the _repeat_..._until_ loop to be able to determine the parameter \(\hat{m}\) more quickly. The given pseudocode is intended to represent the computation in pertinent notation to understand the basic idea for calculation. ```
1:\(\text{CNM}\ S\subseteq U\), \(m=|S|\), \(\alpha\) (signif. level), type\(=\{\text{\emph{Normal}}|\text{\emph{Stud-t}}|\text{\emph{Bootstrap}}\}\), \(\delta^{*}\) or \(\gamma^{*}\) (absolute or relative diff.) 2:\(Val.\ Score_{\text{\emph{prec}}}\). 3:if type\(==Normal\)then\(\triangleright\) Standard Normal Interval
4: Calculate quantile \(z_{\frac{\alpha}{2}}\) according to Standard Normal distribution
5:elseif type==_Stud-t_then\(\triangleright\) Approximation only, Student's \(t\) distribution
6: Calculate quantile \(z_{\frac{\alpha}{2}}\) according to Student's \(t\) distribution with \(m\) degrees of freedom
7:elseif type==_Bootstrap_then\(\triangleright\) Bootstrapping methods
8: Generate \(B\) random bootstrap samples from \(S\subseteq U\)
9: A parameter estimate is calculated from each bootstrap sample
10: Parameter estimates are ordered from low to high \(\triangleright\) If \(\text{\emph{Perc. Conf.