# Mobile Broadband Performance Evaluation: Analysis of National Reports
## 3 Case Studies
### Evaluation Report: Korea  
Regulator\(\rightarrow\) National Information Society Agency (NIA) is responsible for communication services performance evaluation in South Korea, which is referred to as Korea henceforth. The latest report is of 2018 about the evaluation of 2018. NIA has been composing this report since 1999 [11]. Operators\(\rightarrow\) There are three prominent MNOs in Korea. LG U+, SK, KT. Method\(\rightarrow\) Several methods have been used by NIA. First, a professional evaluator moves around in a vehicle and by foot to measure performance through an app installed on evaluation devices. The locations of measurements were statistically sampled to represent the whole country and Galaxy Note 8 was the selected evaluation device in 2018. Second, an RF scanner tool is used to verify the coverage claim of MNOs. Third, a crowdsourcing-based evaluation from a statistically sampled population via an app is performed to understand customer satisfaction. Metrics\(\rightarrow\) Several metrics have been used to evaluate the mobile broadband service. * Connection success rate (SR) \(\rightarrow\) The proportion of calls that attempted to access the measurement server and succeeded. * Transmission success rate (SR) \(\rightarrow\) The proportion of calls that successfully connect to the measurement server and sent the data faster than the minimum speed. * Delay time \(\rightarrow\) The time between signal transmission to the measurement server and receipt of an acknowledgement signal. * Packet loss rate \(\rightarrow\)The proportion of the data that cannot be sent when sending and receiving the data between the device and measurement server of the service provider. * Transmission rate \(\rightarrow\) Data sending and receiving speed between the device and measurement server of the service provider. * Web surfing time \(\rightarrow\) The time required to display the entire web page on the device screen after the user inputs the website address. * Successful connection \(\rightarrow\) Success if connected within 20 s of pressing the call button. * Call drop \(\rightarrow\) Dropped during a call (65 s). * Poor sound quality \(\rightarrow\) If the sound quality value (1-5 points) is below 2.2 or 1.9 more than two times in a row (2.2: difficult to understand, 1.9: impossible to understand). * Over-claim Rate \(\rightarrow\) A proportion of the area where coverage opened by the communications service provider is overstated compared to the actual coverage information. _Evaluated services \(\rightarrow\)_ While voices over 2G, 3G, and 4G services are evaluated, data and coverage are evaluated on the latter two services. _Result \(\rightarrow\)_
* The average download to upload speed stands at 150 Mbps by 44 Mbps, which showed 13 and 44 percent increases, respectively, compared to the year before. While the delay is below 40 ms on average, the loss rate is below 1%. All MNOs have considerably great services in terms of data via 4G/LTE services. This trend is similar for 3G services
* The crowd-sourced evaluation confirms the result obtained by the professional evaluator. The data collected via the app are categorized into five groups (A-E) based on maximum speed. The devices under group A have the same specification as that of the professional evaluator. * The quality of voices is measured while using the same or a different company. The same company refers to the caller and receiver being on the same MNO as well as using the same device types whereas a different company refers to the caller and receiver being on the different MNO as well as using the different device types. Call success rate means the ability to make a successful connection under 20 s, no call drop for \(>\)65, and have a sound quality above 2.2. While the 4G-LTE service provides the highest quality, 2G provides the lowest, which is expected. * While examining the MNOs coverage claim, the report indicated that the average overstatement is 12.9%. LG U+ has the highest overstatement at 34%, whereas SK and KT have 3% and 1% over claim rates, respectively.