## 2 Measuring Mobile Network Quality  
There are different ways how operators can monitor the quality of their network at the user side. This starts with the collection of subjective ratings directly from the end user, continues with monitoring mobile applications and services, and ends with network measurements. In the following, we will present the different methods and also classify and describe the emerging technique of CNMs. **Subjective User Studies** are always required for modeling the user's QoE. Here, people are asked about their satisfaction with a given service under specific network conditions. Using their results, models can be created to identify the key performance indicators (KPIs) of a service or application. These KPIs can later be measured automatically, and can then be mapped to an estimated QoE value. Here, the advantage is that real user experience is included in the evaluation. Nevertheless, this method is very cost-intensive as the participants have to be paid. Furthermore, it is not possible to conduct subjective user studies on a larger scale. Best practices and recommendations for crowdsourced QoE assessment are summarized in [8; 9]. **In-Service Monitoring** is another another way of measuring the networks' quality by passively measure the speed of incoming and outgoing data of an application, for example, a mobile messaging application or smartphone game. In addition, user behavior can be monitored to get deeper insights in the QoE. Negative aspects of in-service monitoring are that the access to use the network information has to be requested and allowed by the smartphone holders. Furthermore, depending on the service in which the measurement tool is included, it is not easy to reach a large number of people, and thus, monitor the mobile network in a large scale for different purposes. **Measurement Applications** are used to monitor the current status of the network by using a standalone measurement application, which can be freely downloaded by smartphone holders who are interested in network statistics. This kind of application offers the possibility to the user to run network speed tests at any time they want to evaluate the current network conditions. In addition, it is also possible to start small network tests at regular intervals, for example daily, to to receive continuous information. Disadvantages of this way of collecting network data is that it is hard to get results on a large scale, as the incentive for the users is limited. It follows that only interested users download this app, and thus, only network statistics from them are collected. The users therefore rather reflect a nonrepresentative group of the population. Furthermore, using measurement apps, only QoS parameters can be monitored; the user satisfaction (QoE) can only be estimated using QoE models. **Hybrid Applications** combine advantages of in-service monitoring and measurement applications. Here, different applications like smartphone games or messaging services can trigger active measurements in addition to passive monitoring. This is especially interesting if the same service provider can address different target groups, e.g., people who play online games and people who use messaging services, to collect QoS values from a heterogeneous group of people. Using different apps, especially if widespread applications cooperate with network measurement companies, it is relatively easy to monitor the mobile network in a large scale. **In-network Measurements** are probably the simplest measurement method for Internet service providers. Here, providers do network measurements within their own network. The biggest advantage of this way of collecting network data is that the measurements can be completed fully automated, and the status of the whole network can be evaluated at regular intervals. Nevertheless, this is also the biggest disadvantage, as only statistics from one provider can be collected, and thus, no comparison of different providers is possible.