# Mobile Broadband Performance Evaluation: Analysis of National Reports
## 5 Discussion and Conclusions  
In this section, a summary of the case studies, discussion, and conclusion is presented. Most of the regulators started performance evaluations in the late 1990s and early 2000s, except for Japan, which started earlier than most, Table 2. However, the Japanese evaluation is mostly non-technical. While Australia measures the performance of mobile broadband as a whole, the US only provided 4G measurements. On the other hand, countries such as Korea and the UK conducted separate and combined (Comb.) performance evaluations. In most cases, while 2G and 3G are for voice and SMS, 4G is used to deliver data services. None of these countries in this study conducted a 5G performance evaluation. Although the reports were supposed to drive the improvement of mobile broadband services, the starting year of the compilation of reports has no correlations with the current rank of the nations. On the other hand, the fact that most of these nations are working to provide real-time service assessment via web apps is encouraging. However, access to the web apps is not straightforward, and there is a lack of awareness by the public, which regulators need to address. Five of the eight nations have three MNOs, whereas the rest have four MNOs (Table 3). Only one, Vodafone MNO, provides services in three different countries. Although Telstra is present in Singapore, its portion of the market and coverage is insignificant. The rest of the MNOs provide service only in a single country. This indicates, the geographical area and population size of the country do not have an impact on the number of MNOs per country. In addition, countries still prefer local MNOs, this is due to the inherent privatization of what used to be national MNOs [33]. \begin{table}
\begin{tabular}{l l l l l l} \hline \hline  & **Regulator** & **Evaluated Service** & **Starting Year** & **Last Year** & **Recent Report** \\ \hline Australia & ACMA & Comb. & 2005 & 2019 & 2020 \\ France & Arcep & 3G, 4G, Comb. & 1997 & 2019 & 2018 \\ Germany & Bundesnetzagentur & Comb., 4G & 2000 & 2018 & 2020 \\ Korea & NIA & 2G, 3G, 4G, Comb. & 1999 & 2018 & 2018 \\ Japan & MIC & 3G, 4G, Comb. & 1993 & 2018 & 2019 \\ Singapore & IMDA & 3G, 4G & 1990 & 2020 & 2019 \\ UK & OffCom & 2G, 3G, 4G, Comb. & 2004 & 2019 & 2019 \\ US & FCC & 4G & 2000 & 2018 & 2020 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: History of mobile broadband performance evaluations. There are several ways to obtain data for performance evaluation. Most nations used coverage and services data provided by MNOs or surveyed the public (Table 4). However, Korean and Singapore regulators conducted a field survey by professional evaluators, which measured metrics such as the received signal strength and call success rate. After obtaining data, some nations applied statistical correction to account for errors. The UK is the only nation that included predictions based on past data in its evaluation. Additionally, the data from third-party performance testers are also used in some cases. We believe, a combination of these techniques is necessary to provide an accurate assessment. Furthermore, we recommend obtaining data from public surveys, MNOs servers, and professional evaluators, which can be fed into a machine-learning algorithm to obtain better insights [34]. There is no standard broadband coverage evaluation, rather each nation provided its own version (Table 5). Some nations consider coverage as a percentage of the population receiving mobile broadband services; others define coverage where the signal is received (blanket coverage). All nations provide what percentage of the consumer receives mobile broadband coverage by at least one MNO. While the rest of the nations provided coverage from all MNOs, Australia and Japan did not disclose this figure, which is also true for coverage reports from each MNO. A very detailed coverage report such as indoor, outdoor, rural, or urban areas is only available on the UK and France reports.