Modeling Quality of Experience of Internet Video Streaming by Controlled Experimentation and Machine Learning. (Modélisation de la qualité d'expérience de la vidéo streaming dans l'internet par expérimentation contrôlée et apprentissage machine)

Video streaming is the dominant contributor of today’s Internet traffic. Consequently, estimating Quality of Experience (QoE) for video streaming is of paramount importance for network operators. The QoE of video streaming is directly dependent on the network conditions (e.g., bandwidth, delay, packet loss rate) referred to as the network Quality of Service (QoS). This inherent relationship between the QoS and the QoE motivates the use of supervised Machine Learning (ML) to build models that map the network QoS to the video QoE. In most ML works on QoE modeling, the training data is usually gathered in the wild by crowdsourcing or generated inside the service provider networks. However, such data is not easily accessible to the general research community. Consequently, the training data if not available beforehand, needs to be built up by controlled experimentation. Here, the target application is run under emulated network environments to build models that predict video QoE from network QoS. The network QoS can be actively measured outside the data plane of the application (outband), or measured passively from the video traffic (inband). These two distinct types of QoS correspond to the use cases of QoE forecasting (from end-user devices) and QoE monitoring (from within the networks). In this thesis, we consider the challenges associated with network QoS-QoE modeling, which are 1) the large training cost of QoE modeling by controlled experimentation, and 2) the accurate prediction of QoE considering the large diversity of video contents and the encryption deployed by today’s content providers. Firstly, QoE modeling by controlled experimentation is challenging due to the high training cost involved as each experiment usually consumes some non-negligible time to complete and the experimental space to cover is large (power the number of QoS features). The conventional approach is to experiment with QoS samples uniformly sampled in the entire experimental space. However, uniform sampling can result in significant similarity in the output labels, which increases the training cost while not providing much gain in the model accuracy. To tackle this problem, we advocate the use of active learning to reduce the number of experiments while not impacting accuracy. We consider the case of YouTube QoE modeling and show that active sampling provides a significant gain over uniform sampling in terms of achieving higher modeling accuracy with fewer experiments. We further evaluate our approach with synthetic datasets and show that the gain is dependent on the complexity of the experimental space. Overall, we present a sampling approach that is general and can be used in any QoS-QoE modeling scenario provided that the input QoS features are fully controllable. Secondly, accurate prediction of QoE of video streaming can be challenging as videos offered by today’s content providers vary significantly from fast motion sports videos to static lectures. On top of that, today’s video traffic is encrypted, which means that network operators have little visibility into the video traffic making QoE monitoring difficult. Considering these challenges, we devise models that aim at accurate forecasting and monitoring of video QoE. For the scenario of QoE forecasting, we build a QoE indicator called YouScore that quantifies the percentage of videos in the catalog of a content provider that may play out smoothly (without interruptions) for a given out- band network QoS. For the QoE monitoring scenario, we estimate the QoE using the inband QoS features obtained from the encrypted video traffic. Overall, for both scenarios (forecasting and monitoring), we highlight the importance of using features that characterize the video content to improve the accuracy of QoE modeling.