# A Hierarchical Classification Model of QoE Influence Factors

Lamine Amour, Sami Souihi, Said Hoceini, Abdelhamid Mellouk

► To cite this version:

Lamine Amour, Sami Souihi, Said Hoceini, Abdelhamid Mellouk. A Hierarchical Classification Model of QoE Influence Factors. 13th International Conference on Wired/Wireless Internet Communication (WWIC), May 2015, Malaga, Spain. pp.225-238, 10.1007/978-3-319-22572-2_16. hal-01728795

HAL Id: hal-01728795

https://inria.hal.science/hal-01728795

Submitted on 12 Mar 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A Hierarchical Classification Model of QoE
Influence Factors

Lamine Amour, Sami Souihi, Said Hoceini and Abdelhamid Mellouk

Networks & Telecommunications Department and LiSSi Laboratory - IUT C/V,
University of Paris-Est Créteil VdM,

Email: (lamine.amour, sami.souihi, hoceini, mellouk) @u-pec.fr

**Abstract.** Quality of Service (QoS) optimization are not sufficient to ensure users needs. That's why, operators are investigating a new concept called Quality of Experience (QoE), to evaluate the real quality perceived by users. This concept becomes more and more important, but still hard to estimate. This estimation can be influenced by a lot of factors called : Quality of Experience Influence Factors (QoE IFs). In this work, we survey and review existing approaches to classify QoE IFs. Then, we present a new modular and extensible classification architecture. Finally, regarding the proposed classification, we evaluate some QoE estimation approaches to highlight the fact that categories do not affect in the same the user perception.

**Keywords :** Quality of Experience (QoE), Mobile environment, Quality of Experience Influence Factors (QoE IFs), Quality of Service (QoS)

## 1 Introduction

Recently, computer networks shift from traditional paradigms to a new one named : Human-centric networks. In this kind of networks, traditional monitoring and management approaches based only on Quality of Service (QoS) optimization are not sufficient to ensure user's needs. That's why, operators are investigating a new concept called Quality of Experience (QoE), to evaluate the real quality perceived by users. The term of the Quality of Experience has been proposed by Dr. D. Norman in 1986, in the context of designing a user-centered services [9]. Several metrics, called Quality of Experience Influence Factors (QoE IFs)[7], can affect the perceived quality by the user. These factors are closely related to human perception and could potentially serve as more valuable quality indicator for all system's actors (user, service and network provider,...). From the users side, it ensures to have the best perceived service regardless their mobility and their context. From the providers side, the QoE helps them to provide, restore and ensure the best service to their users, into decrease the churn rate and increase their benefit.

Although many works addressing QoE concept have been proposed in the literature, this concept is still hard to estimate. For example, in the case of mobile environments, the QoE is used in several contexts (Video on Demand
(VoD), Gaming, Voice over IP (VoIP), ...). In each one of these, QoE IFs (network parameters, application parameters, localization, context, ...) impacting it[4, 7, 3]. That's why, to try to deal with the QoE estimation issue, a fundamental methodological reorientation is needed. To address this issue three steps were required. i) Find all QoE IFs; ii) propose a modeling architecture to organize them; iii) propose an adaptive QoE estimation method which ensure the real perception for the user. In this paper we focus on the first two steps. In section 2, we present an overview of a set of modeling QoE IFs. In section 3, we detail our contribution by explaining the proposed modular and hierarchical architecture in order to organize these factors. Then, we compare the discriminant versus the non discriminant aspect in order to evaluate the importance of the impact of each QoE IFs category on the user perception. Finally, we conclude our work by giving some perspectives.

## 2 Related work

With a telecommunication's industry progress, a rapid technological development in networks had been made. This development allows user to use a lot of services in different context (VoD, Gaming, VoIP, ...). It's implying also that the user is influenced by a huge number of parameters in addition to their specific factors (level studies, gender, knowledge... ). These parameters can be provided by different system entities such as service provider, network operator and/or the user itself.

Ickin et al. [7] call all these parameters : Quality of Experience Influence Factors (QoE IFs), and define them as "any characteristic of a user, system, service, application, or context whose actual state or setting may have influence on the Quality of Experience for the user". To present these factors, many works have been proposed to classify the QoE IFs. In this section, we will present some of them :

- Stankiewicz and Jajszczyk [17] classified the QoE IFs into three groups : QoS factors, Grade of Service (GoS) factors, and Quality of Resilience (QoR) factors. The authors underline that the implementation of each factor in the appropriate group is crucial for achieving high QoE.

- Matulin and Š. Mrvelj [11] presented an evaluation of the factors affecting QoE in four levels : *Core network* is the first level, which can be measured by the parameters of quality of service. The *access network* is the second one. Authors explain that to obtain an accurate level of user perception desired, a minimum access network performance levels must be ensured. The third level is the *hardware quality*. It presents Quality of design (interfaces), perception, service price, security and reliability. The last level concerns the *user*. It is affected by various psychological measures, as the previous user experience, the expectations, the emotional states (status, feelings) and other parameters that must be scrutinized by subjective tests (usually through surveys).

- Ickin et al. [7] organize the QoE IFs into two parts : *applications* and *contexts*. In the first part, the authors have collected QoE factors in thirteen categories based on mobile applications (e.g : Android standard, Web or e-mail applications). In addition, they also mention : Application Interface's
Design, Application Performance and Battery. In the second part, they describe the user context and the conducted experimentation has been done. The applications were used of 80% of the time (in home, in the office and in the school).

- Callet et al. [4] organize QoE IFs into three categories, namely *Human IF*, *System IF* and *Context IF*. The authors define *Human IF* as any variant or invariant property or characteristic of a human user. It can indicate the demographic and socio-economic background, the physical and mental constitution, or the user's emotional state. *Human IFs* are complex, since they refer to the inner world of the individual, their ideas, their background, their feeling and sensations. They explain also that *System IFs* refers to properties and characteristics that determine the technically produced quality of an application or service as: coding, transmission, storage, rendering, reproduction/display and ROM (Read Only Memory) content. Moreover, they explain also that *Context IFs* refer to:

- spatial aspects : such as the location and space characteristics,

- temporal aspects : such as the time of day,

- economic aspects : such as the device cost and telephone subscription price,

- social aspects, such as the presence of other people in the same hall during a video conference.

- Song et al. [16] summarize seven frames by dividing all parameters in three categories, as follows :

- User : Emotion, Needs, Prior experiences, Perceptions, Expectations Motivation, Profile (age, sex, preference, skill/knowledge), Physical resources).

- Product/System/Service : Product appearance, Complexity, Functionality, Usability, Aesthetic quality, Interactivity).

- Context : Context of use, Physical context, Social context, Culture context, Temporal and task).

- Using the same summarization as the last one, Brooks et al. [3] resume seven services (Data conferencing, Real-time text, Multimedia conferencing..etc) to give off a bucking with six categories :

- User choice (Reliability, Cost-benefit...).

- Technical parameters (Delay, Packet loss, Frame-rate).

- User appearance (Head only or Head-and-torso, Eye contact).

- Group communication (Multi-point communication, Window configuration).

- Purpose of communication (Business communication, Decision making).

- User groups (Using a foreign language, Elderly persons at home).

- Moor et al. [12] propose a modular architecture based on layers using web agents. This architecture consists of four layers : *QoS monitoring entity* (Device, Infrastructure, Network, Application), *contextual monitoring entity* (Location, mobility, sensors, other running applications), *experience monitoring entity* (feedback) and *user module* (rules, experiences). The authors indicate that the parameters of a third layer are explicitly obtained while the other layers are obtained implicitly. This model has the advantage that it's a robust measurement approach that allows the modeling of QoE IFs for
mobile multimedia applications. In addition, it is easy to implement and it
combines the objective and subjective experimental components.

According to Balachandran et al. [2], despite this broad consensus, our understanding of Internet video QoE is limited. The author explained that the reason is that Internet video introduces new effects with respect to both quality and experience. In fact, traditional methods of quantifying experience through Mean Opinion Scores (MOS) are now replaced by metrics from the SLA (Service-Level Agreement) such as viewing time and number of visits that more directly impact content provider's business objectives. The study highlights the complex relationship between these parameters and the decision to drop the viewing of the video in *NetFix* provider.

3 New modular architecture QoE Influence Factors
(MaQoEiFs)

There are many publications in the literature that address the problem of modeling parameters affecting user perception (QoE IFs). In the section 2, we have given some of them. Despite of these works, we focus our proposal in this paper to propose a generic model for mobile applications.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Hierarchical architecture classifying QoE influence factors (QoE IFs).</figcaption></figure>
Our proposal is based on a new hierarchical and modular architecture to classify QoE IFs. This architecture considers a large number of QoE IFs and it can be extended according to the future expectations of system's actors. In fact, it can model the various service's factors used (real-time services, IPTV, VoD, social networking, gaming, ...), and different communication technologies types (User centric network, Centric network information, Internet of Thing, Spontaneous networks ...). Furthermore, it takes account a new metrics from the SLA engagement measures such as viewing time and number of visits. Our classification is composed of three layers : Monitoring entity, Contextual monitoring entity and Experience monitoring entity. In addition, each layer is composed of modules as shown in figure 1. Other way, a key aspect of our work is also the classification method of the QoE IFs into subcategories (QoD, QoS, QoA ... etc.) presented in section 3.2.

**3.1 Monitoring entity**

This first layer contains four modules which are described as follows :

**Terminal parameters** : With the rapid advances in technology, there has been a proliferation of new multimedia services. In fact, different terminal types are deployed such as computers, smart phones, tablets, TV and PDA, which also differ from each other by several parameters : screen resolution, processor, SDK version, ...

**Infrastructure parameters** : Network providers may evaluate the subscriber QoE using QoS parameters, but also taking into account information related to the used infrastructure such as : *access type*, *cell's loading rate*, *telephone subscription prices*, ...

<figure><img src="image_3.png" /><figcaption>Fig. 2. Monitoring entity.</figcaption></figure>

**QoS parameters** : QoS parameters reflect the network state and have direct impact on the QoE. Their degradation necessarily involves a bad user perception.
**Applications parameters :** The huge number of services and applications implies different properties and characteristics that have a significant impact on the QoE. As an example, we can note : i) video streaming affected by *the buffer* and *the codec type*. ii) web service influenced by *web page downloading time* and *Http sessions number*.

**3.2 Contextual monitoring entity**

This second layer covers all contextual information about the user. It is composed of four parts :

**Location parameters :** User can easily be located through GPS coordinates, cell identifier or the cell location (x, y, z).

**Mobility parameters :** In the community of ubiquitous and context management, mobility remains an important research field. The estimation of this parameter can be based on terminal characteristics as sensor acceleration and terminal velocity.

<figure><img src="image_4.png" /><figcaption>Fig. 3. Contextual monitoring entity.</figcaption></figure>

**Network parameters :** To evaluate the QoE, various information regarding network can be collected : operator name, network type or network BSSID identifier.

**Psychological Factors :** The user's perception can be different from one person to another, and it depends on several factors that are related to his experience, his environment and preferences. In this context, we can characterize the parameters that are set as follows :

- User characteristics (knowledge, experience, language, age, gender, ...).

- User behavior (degree of interest for the content).

- Physiognomy characteristic (Galvanic skin response).

- Sensors (temperature places, luminosity ...).
## 3.3 Experimentation monitoring entity

This layer shows the experimental methodology and collecting feedbacks. In the case of subjective measurements. It lets towards gathering explicit feedback in the form of questionnaires and pictograph feedback (e.g. pushing a red button if things go wrong).

<figure><img src="image_5.png" /><figcaption>Fig. 4. Experimentation monitoring entity.</figcaption></figure>

## 3.4 Categories

Let's define a set of notations used to present our QoE IFs categories. The developed notation is defined according to the source parameters (System's actors : user, service provider, network operator...) as shown in the following figure:

<figure><img src="image_6.png" /><figcaption>Fig. 5. Sample dataset presentation.</figcaption></figure>

Where :

- QoA : Related to Application parameters.

- QoS : Related to Infrastructure and QoS parameters.

- QoD : Related to Device parameters (Device module, location, mobility and network information modules).

- User Profile : Related to Psychological factors or the human factors.

- User FeedBack : Related to information collected from the experimenta-
tion entity (the user answers some questions that will be useful).

- MOS (Mean Opinion Score) : Represents score given by user for the global quality of the service (application).

This notation allows us to obtain as much as possible the influence degree of each category on the MOS.
# 4 Experimentation

## 4.1 QoE estimation used methods

To evaluate our proposed approach of QoE IFs classification, we conducted two evaluations using six QoE estimation methods based on [2] [13] [8], shown in Table 1. The first one is a non discriminant evaluation (considering all QoE IFs) and the second one is a discriminant evaluation to highlight the fact that categories do not affect in the same the user perception.

<table><caption>Table 1. QoE estimation used methods.</caption><thead><tr><td>Method</td></tr></thead><tbody><tr><td>Native Bays (NB) [13]</td></tr><tr><td>Decision Trees (DT) [15]</td></tr><tr><td>Random Forest (RF) [10]</td></tr><tr><td>Support Vector Machine (SVM) [6]</td></tr><tr><td>Random Neural Network (RNN) [1]</td></tr><tr><td>Adaptive Neuro Fuzzy Inference System (ANFIS) [14]</td></tr></tbody></table>

### a) Naive Bayes (NB)

Bayes (NB) classifier is based on Naive Bayes theorem. It simplifies learning assuming that the features are independent for a given class. With Naive Bayes, the parameters for each term can be learned separately. Furthermore, it simplifies and speeds up the calculation operations. Note that parameter estimation for naive Bayesian models can be based in some cases on the maximum likelihood. [13].

### b) Decision Tree

A decision tree is a decision support tool that is based on the representation of the problem in tree form. The different results are represented by the leaves. This algorithm is characterized by legibility, execution speed and the limited number of needed hypothesis. One of the most used implementation is the "C4.5" proposed by John Ross Quinlan in 2003. [15].

### c) Random Forest (RF)

Random Forests (RF) is a classifier that uses multiple decision trees to improve prediction results. It combines the random subspaces concepts and modern statistical inference methods (bagging). In RF classification, the input vector is submitted to each RF tree for a vote (choose his class). The selected class is the one with the highest number of votes. [10].

### d) Support Vector Machines

Support Vector Machines (SVM) is a classification method, used to solve the
two-class-pattern recognition problem. It analyzes the data and tries to identify patterns so that a classification can be done. The main idea is to find the optimal separating hyper-plane between two classes, by maximizing the margin between the closest points of these two classes [6].

### e) Random Neural Networks

Random neural networks (RNN) are a variant of neural networks. In RNN, neurons may not function continuously, but exchange signals periodically. These signals can be either excitatory or inhibitory: excitatory, they will increase the potential of the neuron that receives them; inhibitors, they are going to reduce this potential. If this potential is positive, then the neuron may itself transmit signals [1].

### f) ANFIS (Adaptive Neuro Fuzzy Inference System (ANFIS))

ANFIS is a Fuzzy Inference System (FIS), which membership function parameters are adjusted using either a back propagation algorithm or a combination of this latter and a least squares method. This method is based on Takagi-Sugeno fuzzy inference system and it integrates both neural networks and fuzzy logic principles proposed by Zadeh in 1965 [14].

## 4.2 Dataset

For all the experiments shown in this paper, we used a dataset collected in our laboratory. It concerns the video service where the MOS value for each sample was calculated using the ACR (Absolute Category Rating) according to the recommendations of the ITU-T (Telecommunication Standardization Sector : forth in ITU-T recommendation P.910). In this experiment, each user has tested 81 combinations. All members were students or researchers from different disciplines aged 17 to 40 years with few or no experience of this kind of evaluation.

<table><caption>**Table 2.** Overall tested conditions.</caption><thead><tr><th>Parameter</th><th>Description</th></tr></thead><tbody><tr><td>Video</td><td>-Types : High motion, average motion and slow motion<br>-Duration : 10 secondes<br>-Video resolution : 500x300<br>-Video codec : 360p (SD)</td></tr><tr><td>Device</td><td>-Types : Smartphone, Laptop and Netbook</td></tr><tr><td>Send Bit Rate (kbps)</td><td>600, 800, 1000</td></tr><tr><td>Frame Rate (fps)</td><td>10, 15, 30</td></tr><tr><td>Link Bandwidth (Mbps)</td><td>1, 1.5, 2</td></tr><tr><td>Packet Loss Rate (%)</td><td>0, 5, 10</td></tr><tr><td>Jitter (ms)</td><td>0, 5, 10</td></tr></tbody></table>

This database contains 243 samples with 9 parameters : device type (QoD), device resolution (QoD), screen (QoD), video content (movement level) (QoA),
send bit rate (QoA), video frame rate(QoA), bandwidth (QoS), jitter (QoS) and packet error rate (QoS). Table 2 summarize the tested conditions.

In fact, two studies were performed in this experimentation. In the first one, we evaluate the impact of our classification on the user perception including all parameters (Non discriminant study). In the second one, we evaluate the impact using just one of the QoE IFs categories (Discriminant study).

These evaluations are performed under Weka [18] or R software[5]. These tools have produced many outputs providing information about the estimation and the error rate calculating.

## 4.3 A non discriminant study :

The first evaluation concerns a non discriminant study (considering all QoE IFs parameters). We measure the Mean Absolute Error rate (MAE) (shown in equation 01) for each studied QoE estimation method (shown in table 1) and compare them.

$$MAE = \frac{1}{n} \sum_{i=1}^{n} |f_i - y_i| \qquad (1)$$

where : $f_i$ is the prediction of MOS, $y_i$ is the true value of the MOS and $n$ is the total number of the considered samples. The results are illustrated in the table 3.

<table><caption>**Table 3.** Mean Absolute Error (MAE) -All parameters-.</caption><thead><tr><th>Method</th><th>NB</th><th>DT</th><th>RF</th><th>SVM</th><th>RNN</th><th>ANFIS</th><th>Average</th></tr></thead><tbody><tr><td>MAE</td><td>0.1392</td><td>0.1156</td><td>0.1170</td><td>0.1119</td><td>0.1156</td><td>0.1143</td><td>0.1195</td></tr></tbody></table>

As we see, there are not a huge difference among the error rate of these methods, where the difference between the best estimation method and the less one is : 2%. The results are illustrated in the figure 6.A (All QoE IFs – QoE)

<figure><img src="image_7.png" /></figure>

Despite these estimation methods have acceptable results (approximately), using QoE IFs categories without weighting can produce a worse estimation result then expected. In fact, QoE IFs are inter-dependent. For example, according to [2], streaming video at a higher bitrate would lead to better quality. However, it would take longer for the video player buffer to sufficiently fill up in order to start playback. In order to improve the QoE estimation we need to study the importance of each QoE IFs category on the estimation and their interdependence. That's why, we will address separately the impact of the QoE IFs categories (Discriminant study) in the next section.

## 4.4 A discriminant study :

Figure 6 (B, C and D) represents the obtained results by the different studied estimation methods considering one category at a time ( summarized in Table 4).
<table><caption>**Table 4.** Mean Absolute Error (MAE) -QoD/QoS/QoA-.</caption><thead><tr><th>Method</th><th>NB</th><th>DT</th><th>RF</th><th>SVM</th><th>RNN</th><th>ANFIS</th><th>Average</th></tr></thead><tbody><tr><td>QoD</td><td>0.2209</td><td>0.2224</td><td>0.2232</td><td>0.1514</td><td>0.2213</td><td>0.1917</td><td>0.2052</td></tr><tr><td>QoS</td><td>0.1480</td><td>0.1400</td><td>0.1288</td><td>0.1004</td><td>0.1300</td><td>0.1227</td><td>0.1283</td></tr><tr><td>QoA</td><td>0.1622</td><td>0.1681</td><td>0.1601</td><td>0.1284</td><td>0.1624</td><td>0.1295</td><td>0.1518</td></tr></tbody></table>

As shown these figures, the SVM estimation gives better results than others studied methods in term of Mean average error independently of the QoE IF category ( where the MAE of the SVM presents : 12% compared to ANFIS = 14%, RNN = DT = RF = 16% and NB = 17%). In addition, these figures show that we can estimate roughly the user perception using only one category. Another conclusion is in some cases (QoS category), the obtained results using one category is better than the obtained ones using all categories. This fact, confirms the assumption made by [2] about the interaction between metrics : "Naturally, the various quality metrics are interdependent on each other".

<figure><img src="image_8.png" /><figcaption>**Fig. 6.** Mean Absolute Error (MAE) for the parameter chosen using No discriminant and discriminant study.</figcaption></figure>

## 4.5 Discussion

Except for the SVM approach, using all QoE IFs categories produces a better user's perception estimation than using each one of them separately (average
MAE based on only one QoE IF category : $QoD$ = 25%, $QoS$ = 13% and $QoA$ = 15% versus considering all categories 10%). However, the interaction between parameters may lead to worse results than expected (e.g. SVM).

The experiment allowed us to confirm that the user perception estimation accuracy can be improved by considering a large number of QoE IFs. In addition, these QoE IFs should be weighed according to the context. Our hierarchical classification model takes into account this weighting.

# 5 Conclusion

With the emergence of the human-centric-networks, Quality of service (QoS) is being not enough to evaluate the system's performance. That's why Quality of Experience metrics have being more and more important. In fact, these metrics tends to be widely used as an input to measure the satisfaction of system's actors (user, network and service provider). However, QoE is very hard to estimate. To address the QoE estimation issue, the first step is to determine all factors that influence the user perception commonly called : QoE Influence Factors (QoE IFs) and organize them.

This paper starts by presenting and reviewing some of QoE IFs classifications. Each one of them has some advantages, but remained however suffering from some drawbacks. That's why we propose a new modular architecture to classify QoE IFs. This architecture considers a large number of QoE IFs and can be extended according to the future expectations of system's actors. In addition, it can be used in several user's contexts (Video on Demand (VoD), Gaming, Voice over IP (VoIP), ...). Another key aspect of our work is the classification of the all QoE IFs on several categories (QoD, QoS, QoA ...). To study and compare the impact of these categories on the user perception, we evaluate the impact of our classification on the user perception (Non Discriminant study) and compare them separately (Discriminant study). QoS category is the most impacting the user perception according our experimentation results. Furthermore, using all the QoE IFs categories with the same importance can produce a worse estimation results then expected. As perspective, we continue working on QoE weights and improve the experimentation in order to build bigger and consistent dataset.

# Acknowledgment

This work has been funded by LiSSi laboratory from the UPEC university in the framework of the French cooperative project PoQEMoN, *Pôle de Compétitivité Systematic* (FUI 16).
Bibliography

[1] O. O. Olugbara B. T. Abe and T. Marwala. Hyperspectral image classification using random forests and neural networks. *Proceedings of the World Congress on Engineering and Computer Science 2012 Vol I, WCECS 2012*, San Francisco, USA, October 24-26, 2012.

[2] A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. Developing a predictive model of quality of experience for internet video. *Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM*, pages 339–350, October 2013.

[3] P. Brooks, B. Hestnes, S. Heiestad, and C. Aaby. Communicating quality of experience data for the development of multimedia services. *Proceedings of the 20th International Symposium on Human Factors in Telecommunication, Sophia Antipolis, France*, March 21-23, 2006.

[4] P. L. Callet, S. Moller, and eds.. A. Perkis. Qualinet white paper on definitions of quality of experience (qoe) and related concepts. *Lausanne, Switzerland, Output version of the Dagstuhl seminar 12181, European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003)*, June 3, 2012.

[5] Institute for Statistics and Mathematics of WU. R software. (*Wirtschafts universität Wien*). URL http://www.r-project.org/.

[6] Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. The weka data mining software: an update. *ACM SIGKDD explorations newsletter*, 11(1):10–18, 2009.

[7] S. Ickin, K. Wac, M. Fiedler, L. Janowski, J.Hong, and A. K. Dey. A factors influencing quality of experience of commonly-used mobile applications. *IEEE Communications Magazine*, pages 48–56, 2012.

[8] A. Khan, L. Sun, and E. Ifeachor. Qoe prediction model and its application in video quality adaptation over umts networks. *Multimedia, IEEE Transactions on (Volume:14, Issue: 2)*, pages 431–442, April 2012.

[9] R. Harris L. T. Nguyen and A. Punchihewa. An enhanced analytical framework integrating aspects of network and application performance, psychology, and content in qoe assessment for web browsing. *International Journal of Computer and Information Technology (ISSN: 2279 0764)*, page Volume 02 Issue 05, September 2013.

[10] M. Malekipirbazari and V. Aksakalli. Risk assessment in social lending via random forests. *Expert Systems with Applications Journal*, February 2015.

[11] M. Matulin and Š. Mrvelj. State-of-the-practice in evaluation of quality of experience in real-life environments. *Information and Communication Technology Preliminary Communication*, 255-264, 23 May 2013.

[12] K. De Moor, I. Ketyko, W. Joseph, T. Deryckere, and eds. L. De Marez. Evaluating quality of experience of mobile applications and services in a living lab setting. *Mobile Networks and Applications archive*, Volume 15 Issue 3:378–391, June 2010.

[13] M.S. Mushtaq, B. Augustin, and A. Mellouk. Empirical study based on machine learning approach to assess the qos/qoe correlation. *Networks and Optical Communications (NOC), 2012 17th European Conference on*, pages 1–7, June 2012.

[14] Lala Septem Riza, Christoph Bergmeir, Francisco Herrera, and Jose Manuel Benitez. Learning from data using the r package" frbs". *Fuzzy Systems (FUZZ-IEEE)*, 2014 IEEE International Conference on, pages 2149–2155, 2014.
[15] F. Rodriguez-Teja and E. Grampin. Wekatie, a traffic classification plugin integrating tie and weka. *Wireless Communications and Mobile Computing Conference (IWCMC)*, pages 623 – 628, Aug. 2014.

[16] Wei Song, Dian W. Tjondronegoro, and Michael Docherty. Understanding user experience of mobile video: framework, measurement, and optimization. *Mobile Multimedia: User and Technology Perspectives, INTECH Open Access Publisher*, pages 3–30, January 2012.

[17] R. Stankiewicz and A. Jajszczyk. A survey of qoe assurance in converged networks. *Computer Networks*, Volume 55, Issue 7:1459–1473, 2011.

[18] Weka 3: Data Mining with Open Source Machine Learning Software in Java. URL http://www.cs.waikato.ac.nz/ml/weka/downloading.html.