IEICE TRANS. COMMUN., VOL.E104–B, NO.7 JULY 2021

716

INVITED PAPER

Special Section on Future Directions of Research and Development on Communication Quality

Quality of Experience (QoE) Studies: Present State and Future
Prospect
Tatsuya YAMAZAKI†a) , Fellow

SUMMARY
With the spread of the broadband Internet and highperformance devices, various services have become available anytime, anywhere. As a result, attention is focused on the service quality and Quality of Experience (QoE) is emphasized as an evaluation index from theuser’s viewpoint. Since QoE is a subjective evaluation metric and deeply
involved with user perception and expectation, quantitative and comparative research was difficult because the QoE study is still in its infancy. At
present, after tremendous devoted efforts have contributed to this researcharea, a shape of the QoE management architecture has become clear. Moreover, not only for research but also for business, video streaming services
are expected as a promising Internet service incorporating QoE. This paper reviews the present state of QoE studies with the above backgroundand describes the future prospect of QoE. Firstly, the historical aspectsof QoE is reviewed starting with QoS (Quality of Service). Secondly, a
QoE management architecture is proposed in this paper, which consists of
QoE measurement, QoE assessment, QoS-QoE mapping, QoE modeling,
and QoE adaptation. Thirdly, QoE studies related with video streaming
services are introduced, and finally individual QoE and physiology-based
QoE measurement methodologies are explained as future prospect in thefield of QoE studies.
key words: QoE, QoS, video streaming, user experience, personalization,
physiology

1.

Introduction

Owing to broadband networks and high-performance user
devices, Internet services have become rich and diverse including video streaming, virtual reality, and cloud gaming.
Because they are “services”, concerns of service providers
lie in securing service quality they provide. On the other
hand, customers also evaluate a service by its quality. Therefore, tremendous effort has been provided to measure andimprove the service quality.
Quality of Experience (QoE) has been attracting attention to study the service quality from the user’s point of viewfor ten years or more. QoE is a subjective metric about theoverall service quality reflecting user perception, expectations, and experience for the service [1], [2]. Since QoE
deals with the subjective matters, it is difficult to establish
a generic theory or model. However, since much effort has
been devoted to QoE research so far, a holistic understanding of service quality has progressed and individual research
topics have been clarified. In this paper, QoE measurement,
QoE assessment, QoS-QoE mapping, QoE modeling, andManuscript received September 17, 2020.
Manuscript revised December 25, 2020.
Manuscript publicized February 4, 2021.
†
The author is with the Faculty of Engineering, Niigata University, Niigata-shi, 950-2181 Japan.
a) E-mail: yamazaki.tatsuya@ie.niigata-u.ac.jp
DOI: 10.1587/transcom.2020CQI0003

QoE adaptation are discussed as the research topics.
QoE is also important in the business domain. Under
limited resources, service providers try to deliver appropriate service quality to individual users. When user tolerance for quality impairments is decreasing because of infrastructural limitations, it influences users’ choice in adopting
one service provider over the other [3]. Therefore, service
providers have to focus on QoE to retain customers. The
providers monitor the customers’ QoE and control the QoE
mechanism to prevent the customers from abandoning theirservice because of QoE deterioration.
Since fifth generation mobile communications (i.e. 5G)
has come into practical use and 45% of all networked devices will be mobile-connected by 2023, population of mobile Internet users is expected to increase more. Since theaverage 5G speed will be 575 Mbits/s, video streaming is
becoming one of the most popular services over mobile networks [4]. As a result, according to [5], video streaming
will occupy 82% of all consumer Internet traffic by 2022,
which was 75% in 2017. Thus, video streaming service,especially in a mobile environment, must be a promising Internet service. Service and network providers have to pay
more attention to building services based on QoE. In addition, not just from a business perspective, QoE research
for the video streaming service will be conducted actively
as one of vibrant study areas. To that end, QoE research
field is growing as an interdisciplinary study. Since we haveto grasp user’s satisfaction, feeling, complaint, and so on,
collaboration with other research fields including usability
engineering, psychology, and biometrics must be activatedmore and more [6], [7].
This paper is organized as follows. The historical aspects of QoE are reviewed in Sect. 2. In Sect. 3, QoE research is divided into several individual research topics andrespective topics are discussed. Among various Internet services, Sect. 4 overviews QoE research for the video streaming service as a future popular and promising service. After
future prospect of QoE study is briefly discussed in Sect. 5,
Sect. 6 concludes this paper.
2.

Historical Aspects of QoE

Through discussion on the service quality started in the fieldof telephony at the end of the 20th century, ITU-T (International Telecommunication Union Telecommunication Standardization Sector) defined the terminology “QoS (Qualityof Service)” as “Totality of characteristics of a telecommu-

Copyright c 2021 The Institute of Electronics, Information and Communication Engineers

YAMAZAKI: QUALITY OF EXPERIENCE (QOE) STUDIES: PRESENT STATE AND FUTURE PROSPECT

717

nications service that bear on its ability to satisfy stated andimplied needs of the user of the service” [8]. Although
we notice that “the user needs” were already included in
the definition, QoS has gradually become the metrics tomeasure system (especially network) performance. For example, Karim et al. [9] specified bandwidth, transmission,
packet loss, and delay as QoS metrics for video quality. As
a result, more user-centric concept for the service quality
was introduced as “Quality of Experience (QoE)” in ITUT Recommendation P.10/G.100 Amendment 1 in 2007 [10].
The evolution from QoS is that “Service” has changed to“Experience”. The origin of the term “Experience” is related with “User Experience (UX)”, which is defined in ISO
9241-210:2019 [11], for example, as follows:user’s perceptions and responses that result
from the use and/or anticipated use of a system,
product or service.
Namely, including “Experience” in QoE clearly demonstrates the shift to the user’s perspective. Consequentially,
although QoE comes from the telecommunication field and
UX comes from Human-Computer Interaction field, bothfocus on human perception in common.
According to the above consideration, The EU QUALINET community [12] has defined QoE as follows:
A measure of user performance based on bothobjective and subjective psychological measuresof using an ICT service or product. Quality ofExperience is the degree of delight or annoyanceof the user of an application or service. It resultsfrom the fulfilment of his or her expectations withrespect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.
Additionally, it should be kept in mind that the following
description was given under the definition: “In the context ofcommunication services, QoE can be influenced by factors
such as service, content, network, device, application, andcontext of use”.
On the other hand, Study Group 12 of ITU-T discusses
the definition of QoE continuously. The initial definition of
QoE agreed by ITU-T in 2007 was as follows [10].
The overall acceptability of an application orservice, as perceived subjectively by the end-user.
NOTE 1 Quality of Experience includes the complete
end-to-end system effects (client, terminal,
network, services infrastructure, etc.).
NOTE 2 Overall acceptability may be influenced byuser expectations and context.
Recently, this QoE definition has been replaced as the degree of delight or annoyance of the user of an application orservice [13]. It is partially but completely consistent with

Fig. 1

Relation between QoS and QoE.

the definition of the EU QUALINET community, because
the new definition is affected by discussions in QUALINET.
The review of the QoE definition has, however, been done
constantly in ITU-T, and the above definition is the latest at
the time of writing this paper.
Considering these standardization activities, Yang et al.
[14] summarized that QoE is a kind of subjective perception
generated by users in the interaction process between users
and services or applications, or QoE can be considered as
the overall recognition degree of the employed services oroperations in a certain objective environment. Apparently,
the end user is the decision-maker for service quality.
From the above definitions, QoE can be considered as
an extension of QoS based on a user’s viewpoint. This concept is depicted as Fig. 1(a); QoE is overall evaluation of aservice including QoS elements. On the other hand, QoE
is sometimes considered in contrast to QoS. For example,
Dong et al. [15] grasped QoE as a new concept related tobut differing from QoS perception. This idea corresponds toa concept shown in Fig. 1(b), which has advantages in modeling QoE and QoS. Both of Fig. 1(a) and (b) are correct,
and both concepts are used in a mixed way in this paper.
This is because a unified definition or scope of QoE is notavailable through different communities yet; the situation isalmost the same one as described in [1].
3.

QoE-Related Research Issues

Although a lot of researchers have contributed to QoErelated research, their approaches were different from eachother more or less, since their contribution depends on theirrespective research area such as communicaion network,
media, content, context, and so on [16]–[18]. Moreover,different applications have different QoE requirements (alsoincluding different QoS-dependencies). Thus, there are still
difficulties to realize and deploy a holistic research basement
and common communication services based on QoE.
Nevertheless, through the previous research, the architecture of end-to-end QoE management has been clarified
and the architecture can be divided into several research
topics. This paper focuses on four research topics: QoE
measurement, QoS-QoE mapping, QoE modeling, and QoE
adaptation as shown in Fig. 2. QoE measurement, which iscalled QoE assessment or QoE evaluation in some cases, isthe initial method to evaluate QoE assessment. Next, QoS
parameters and QoE measurement results are associated by
QoS-QoE mapping and QoE models are constructed basedon the QoE factors that influence QoE evaluation and the

IEICE TRANS. COMMUN., VOL.E104–B, NO.7 JULY 2021

718

Fig. 2
Table 1

QoE-related issues.

An example of MOS scale and corresponding quality.
MOS scaleQuality

5
4
3
2
1

Excellent
Good
Fair
Poor
Bad

above-mentioned results. QoS-QoE mapping is also called
QoS-QoE translation. Finally, QoE models are utilized toadapt and guarantee QoE or to control QoS parameters according to the service or system object. This process iscalled QoE adaptation, control, or assurance. This paper
uses the terms of QoE measurement, QoS-QoE mapping,
QoE modeling, and QoE adaptation mainly, and each research topic is discussed one by one in the following subsections.
3.1

QoE Measurement

Returning to the latest QoE definition of ITU-T, it is the degree of delight or annoyance of service for the user [13].
Accordingly, the best way to know the user’s satisfaction
degree is to inquire it of the user.
There are generally two main QoE measurement
methodologies: subjective and objective approaches [2], [3],[19], [20]. The most common subjective approach is the
Mean Opinion Score (MOS), which is defined by ITU-T[21]. As a typical subjective QoE measurement, a group ofusers experience a provided service, evaluate it, and rate theservice on a MOS scale. Common guidelines for conducting subjective measurement are issued by ITU-R (International Telecommunication Union Radiocommunication Sector) [22]. An example of the MOS scale is provided in Table 1.
One of the drawbacks of the subjective approach is that
it is costly and time consuming to gather a number of users.
Moreover, the subjective tests should be experimented in acontrolled laboratory environment.
Zhu et al. [3] pointed out two inconveniences for labbased experiments in addition to costly and time-consuming
points. One is that the demography of the users is often not
representative of the general population, and the other is thatthe laboratory environment often fails to simulate the reallife environment. Subsequently, they suggested Internetbased subjective experiments which utilize Web sites, social media, and/or crowdsourcing platforms [23]. Niida etal. [24] developed Web-based mobile services and studied
the end user perception of the waiting time for Web access.
They conducted Internet-based field experiments to collect
MOS as a metric of subjective quality of user satisfaction.
On the other hand, Seshadrinathan et al. [25] recruited
38 subjects to assess video quality by the difference mean
opinion scores (DMOS). Then they used the subjective
DMOS results to evaluate several objective video quality
assessment algorithms using Peak Signal-to-Noise Ratio(PSNR) and so on. PSNR is used to detect video signal
distortions pixel by pixel. Also, some service quality estimation from several network parameters is another example
and it is related with QoS-QoE mapping in Sect. 3.2.
Another QoS measurement related with the objective
approach was presented by Yamazaki et al. [26]. Since subjects usually evaluate service quality at a stationary time
during or after experiments, it is normally difficult to evaluate time-varying video quality. Yamazaki et al. developed
a tool to record the subjective video quality evaluation continuously during video playing. Simultaneously, the user’s
objective quality evaluation was recorded by physiologicalmeasures. In their experiments, Skin Conductance Activity (SCA) was measured as the objective quality evaluation.
The value of SCA changes depending on skin sweating andis suitable to measure the degree of psychological stress.
They recorded QoE and SCA data for video streaming services simultaneously. The video streaming was deteriorated
by abrupt playback pause. After the measured QoE andSCA signal data are analyzed by the two-way analysis ofvariance, it has been clarified that they are correlated anddifferentiated by the video category and the playback pause
period.
Through exhaustive survey of QoE research, it is foundthat QoE assessment is used with exactly the same meaning
as QoE measurement. For example, Huang and Ishibashi[27] and Hoßfeld et al. [28] used QoE assessment instead of
QoE measurement. Barman and Martini [29] referred QoE
assessment as the process of measuring or estimating the
QoE and described that the main objective of QoE assessment is the design of a system which can identify the various factors and their influence on the end user QoE. In thisway, one idea is that the role of QoE assessment should be
judgement for the user QoE, whereas QoE measurement isjust to collect QoE data from the users.
3.2

QoS-QoE Mapping

Based on the concept shown in Fig. 1(b), it is assumed thatuser QoE can be separated from the QoS parameters that
demonstrate system performance. The QoS parameter examples are packet loss rates and delays in the network layer
and frame rates and resolutions in the application layer. The
QoS-QoE mapping means relationship between the QoS parameters and QoE evaluation. The QoS-QoE mapping issometimes called the QoS-QoE translation. Hoßfeld et al.
called their method as QoS-to-MOS mapping because theyused MOS rating as QoE evaluation [30]. Although the

YAMAZAKI: QUALITY OF EXPERIENCE (QOE) STUDIES: PRESENT STATE AND FUTURE PROSPECT719

QoS-QoE mapping can be bi-directional in principle, it isoften used to provide QoE evaluation from available QoSparameters.
Before the term “QoE” arises, several layered QoS
models were proposed to separate the user side from thesystem side and an interface between both sides was defined[31]–[33]. The function of the interface connecting the userand system sides was called QoS mapping.
As the conventional QoS mapping methods were categorized into two classes [34]: a table-based mapping (e.g.,[35]) and a function-based mapping (e.g., [36]), the QoSQoE mapping also have these two categories. The tablebased mapping makes use of QoS mapping tables of sample data which are prepared in advance. On the other hand,the function-based mapping uses mathematical functions formapping that derive an appropriate output for an input; none
the less, the challenge is how to determine the type or the dimension of the function. In effect, the function-based mapping is major in these days.
The function-based QoS-QoE mapping method normally derives a mapping function with QoE as an output
objective variable and the QoS parameters as input explanatory variables. Concrete QoS-QoE mapping instances arepresented in the below.
Benmir et al. [37] selected packet loss rate, jitter, anddelay as the explanatory variables and computed QoE evaluation from these explanatory variables. The formula of computation originated from [38]. Then the computed QoE wasconverted to MOS.
Georgopoulos et al. [39] studied video quality in avideo streaming service. They selected resolution and bit
rate (kbps) in a log scale as explanatory variables. As a corresponding objective variable, they selected the Structural
Similarity (SSIM) index that is a functional model of thehuman visual system. They specified the function type andcomputed its coefficients by matching sample data with thefunction type.
Egger et al. [40] derived a slightly different mapping
function from the above two cases. They studied psychological perspective on the waiting timing and brought over
the Weber-Fechner law. The Weber-Fechner law is a psychological stating that the perceived intensity of a sensation
is proportional to the logarithm of the stimulus. In their research, the waiting timing corresponds to the stimulus andthe perceived intensity corresponds to QoE. They dealt withsimple waiting tasks such as file downloads, where the waiting time was considered as an explanatory variable and theuser’s perception is considered as an objective variable.
Finally, a more recent approach using Machine Learning (ML), that is an ML-based mapping, is introduced.
Menkovski et al. [41] constructed QoE prediction models that determine the user thresholds of acceptability for
the QoS parameters. The QoS parameters were time (s),video bit rate (kbit/s), audio bit rate (kbit/s), and frame rate
(frames/s). Using the datasets, QoE prediction models wereconstructed by two ML methods: the Decision Tree (DT)and the Support Vector Machine (SVM). For their case, theFig. 3

A QoE model - a service quality coordination model.

DT outperformed the SVM. Alreshoodi and Woods also
cited the previous studies of QoE prediction models using
several ML methods including the DT and the SVM [2].
3.3

QoE ModelingIn this paper, QoE modeling is defined as specifying the elements that influence QoE evaluation and clarifying theirmutual relationship. The influential elements for QoE evaluation are called the QoE factors. Although the QoE factors
depend on the issues dealt with and there are a great variety
of QoE factors, some classification of the QoE factors has
been proposed.
Figure 3 presents a QoE model introduced in [42]; In
the original literature, it was called a service quality coordination model. It is a relatively simple and abstract model,
so that it is adaptable for any service situation. The user satisfaction computation mechanism is located in the center ofthe QoE model with the QoS parameters and the QoE factors as its inputs. After the computation, a degree of user
satisfaction for the telecommunication service is provided
as the output. The QoS-QoE mapping function is one of theessential realization of the model.
In Fig. 3, the QoE factors are divided into two types:
observable and unobservable. The observable QoE factors
are defined as the QoE factors that can be observed and parameterized directly. On the other hand, the unobservable
QoE factors are defined as the one that cannot be parameterized easily. For example, the type of content such as video
or audio and the media-related parameters (e.g., frame rates
and resolution) can be specified as the observable QoE factors. Meanwhile, relationship and synchronization among
different media are difficult to be parameterized automatically. Here, synchronization means temporal relationships
between different kinds of media presented to a user. Whenthe presentation timings are identified, these media are considered to be synchronized. Without time stamps, it is normally difficult to be parameterized. In many cases, the QoE
factors are considered as unobservable.
Although there is no unified and decisive classificationof the QoE factors, a few studies indicate the classificationof QoE factors. Barman and Martini [29] cited the QoE factors as “any characteristic of a user, system, service, application, or context whose actual state or setting may have an
influence on the Quality of Experience for the user” [12],IEICE TRANS. COMMUN., VOL.E104–B, NO.7 JULY 2021

720

and categorized them into four categories of Influence Factors (IFs):
1. System IFs which mostly consist of the technical aspects of quality, for example, the ones which can bemeasured using QoS based measurement approaches.
2. Human or User IFs which include aspects referring to
the information about the end user and related aspects.
3. Context IFs which deal with factors such as location,
end user environment, time of the day, type of usage,
time of service consumption.
4. Content IFs which address the characteristics of thecontent.
Skorin-Kapov and Varela [43] built a generic QoE
model called the ARCU (Application-Resource-ContextUser) Model independent of a particular service type. In
this model, they defined four multi-dimensional spaces including the QoE influence factors.
1. Application space A: it is composed of dimensions representing application/service configuration factors.
2. Resource space R: it is composed of dimensions representing the characteristics and performance of the technical system and network resources used to deliver theservice.
3. Context space C: it is composed of dimensions indicating the situation in which a service or application is
being used.
4. User space U: it is composed of dimensions related tothe specific user of a given service.
These four spaces can correspond to the four IFs one by one
in [29].
Zhu et al. [3] also extracted the system factors and thecontext factors as the QoE factors as follows:
System factors
These determine service quality from the system aspect. For example, visible artifacts such as blockiness, blur, and ringing are the factors in the applicationlevel. The network QoS parameters such as buffer ratio, buffering duration, and average bit rate belong tothem.
Context factors
These are the factors that influence user situation. They
entail characteristics of the physical environment, thepresence (in situ or remote) of other users experiencing
the same media, or economic conditions regulating forthe service fruition.
In addition, user factors were also introduced in [3], which
are shortly introduced in Sect. 5.
Focusing on video streaming and with reference to[12], [14], Benmir et al. [44] grouped the QoE factors inthree categories: human, system, and context influence factors (IFs).
1. Human IFs are any property or characteristic of a human. They could be divided into two subgroups. The

first is the low-level IFs such as age, gender, personality and mood. The second is the high-level IFs such associo-economic conditions, educational background,needs, previous experience and life stage. Examplesof Human IFs are gender, age, and educational background.
2. System IFs refer to properties and characteristics that
determine the technically produced service quality.
They could be classified into content-related, mediarelated, network-related, and device-related factors.
Examples of System IFs are delay, loss, and resolution.
3. Context IFs consider the environmental factors associated with the user. Their classification includes six
subgroups: physical factors, time factors, social factors, economic factors, factors associated with task assignments, and technical factors. Examples of Context
IFs are location, cost, and frequency of use.
3.4

QoE Adaptation

After measuring, assessing, and modeling QoE for an Internet service, the final purpose is to provide optimal QoE
for the end users. As seen in the QoS-QoE mapping, QoEtightly correlates with QoS parameters that control system
performance. More concretely the network and application parameters should be adjusted to maximize the end user
QoE within the constraints of available resources, since thegoals of network management are efficient resource allocation typically [45]. Thus, system resource management
based on QoE is called QoE adaptation, QoE control, or
QoE assurance.
Dai [1] advocated QoE-based QoS engineering, where
the service quality was expected to be improved by network
resource optimization. Its target was to maximize the user
experience while reducing network costs concurrently. Although a top-down approach for QoE-based QoS engineering was proposed as an algorithm, there was no substantive
instance.
Georgopoulos et al. [39] introduced an OpenFlowassisted QoE Fairness Framework (QFF). The object of QFF
was to optimize the QoE for all video streaming users, i.e.
to fairly maximize users’ QoE. OpenFlow allows vendoragnostic functionality to be implemented for network management and active resource allocation. They constructeda home network environment testbed, where an OpenFlow
switch connected several PCs that imitated user devices in
the home network environment. The home network wasconnected to an HTTP Web server via Internet. The experimental results demonstrated that QFF provided network
stability and optimized video streaming QoE across threeheterogeneous user devices in the home network environment.
Kwon et al. [46] proposed an adaptive mobile VoIP
(mVoIP) service architecture based on network QoS andmVoIP QoE data in SDN (Software Defined Network) networks to improve mVoIP QoS. They presented two key ap-

YAMAZAKI: QUALITY OF EXPERIENCE (QOE) STUDIES: PRESENT STATE AND FUTURE PROSPECT

721

proaches: flow path optimization for mVoIP traffic using theSDN controller and the adaptive codec selection method.
However, only an algorithm was offered to realize the proposed architecture.
Huong Pham-Thi et al. [47] also proposed a bandwidth
allocation method considering user characteristics. In thisresearch, they classified users into different groups based on
their characteristics such as relaxed or busy. Then different
utility functions, that is QoS-QoE mapping functions, areassigned for each user group differently. Under the resource
limitation, the bandwidth of the sharing network was properly assigned for the user groups respectively.
4.

QoE-Based Video Streaming

As described in Sect. 1, video streaming traffic, particularly in mobile, is growing in volume. Continuously, video
streaming is expected to expand on the business side. Therefore, video streaming services have been the focus of attention as the service to introduce the QoE adaptation.
Before the popularity of video streaming increased,
standardization activities progressed for objective quality
assessment of audio and visual media, IPTV services [48].
The media-layer models in the standardization activities are
just briefly referred to in this paper. There are three objective video quality assessment methods: full-reference (FR),reduced-reference (RR), no-reference (NR). FR methods extract the quality difference by comparing the source reference video signal and its processed counterpart, assumingthat the undistorted reference signal is fully available. RR
methods extract some features of the source reference andits counterpart videos and compare them to offer a quality
rating. NR methods assess the quality of a processed or distorted video without any reference signal. Due to the absence of a reference signal, video quality assessment must
be more difficult than FR or RR.
Considerable and rapid progress for QoE-based video
streaming services is also observed in the area of researchand development.
Zhu et al. [3] were motivated to devise accurate tools tomeasure the extent to which the user deems the multimedia
experience to be of a high quality, because resource scarcity
makes flawless delivery more challenging. Consequently,
they developed a Facebook application YouQ, for the collection of individual video QoE data. These are multiple
advantages of YouQ.
• It can reach users with diverse demographics in reallife viewing environments.
• It can collect user information automatically fromFacebook.
• It allows checking for user reliability at every stage ofthe experiment.
• It is open-sourced for research. Researchers are free toedit the set of stimuli and questionnaires based as needbe.
Zhu et al. have more interests for “individual QoE” that is

discussed in Sect. 5.
Recently, HTTP adaptive streaming (HAS) has gainedmuch attention [29]. HAS is currently used by the major video streaming services such as Netflix and YouTube.
Therefore, many research works have focused on developing QoE models targeting HAS-based applications.
Also, The ITU-T Recommendation series P.1203, known as
P.NATS, describes a parametric bitstream-based model forthe quality of progressive download and adaptive streaming
based applications over a reliable transport protocol such as
TCP [49]. In this context, Yamagishi and Hayashi [50] presented a quality model which was submitted as part of thecompetition for the ITU-T Recommendation series 1203.
Video streaming services are discussed as a mobile environmental service. Benmir et al. [37], [44] discussed onvideo streaming services over Vehicular Ad-hoc Networks(VANETs). VANETs are a class of Mobile Ad-hoc Networks (MANETs) [51] and are designed for communication
between vehicles (V2V) or between vehicles and road infrastructure (V2I) [52]. One of the promising services implementing on VANETs is video streaming services for traffic safety or entertainment.
Firstly, Benmir et al. [44] surveyed a large number ofprevious works and concluded that the previously proposedmodels had many limits and drawbacks. For example, few
proposed models were dealing with QoE over VANETs, therelationship between QoE and its influencing factors wascomplex and nonlinear, and only a limited number of influencing parameters were considered in the previous models.
Then, Benmir et al. [37] proposed a QoE-aware geographic protocol for video streaming over VANETs. In
the proposed protocol, the selection process of the next relay vehicle was based on a correlated formula of QoE andQoS factors to enhance the users’ QoE. They evaluated the
proposal by computer simulation. The simulation results
showed that the proposed protocol outperformed two existing protocols in providing the best user QoE of video
streaming service.
5.

Future Prospect

As future prospect, two research topics in QoE measurement
are considered. One is individual QoE, or personalize QoE,
and the other is physiology-based QoE measurement.
MOS is one of the popular schemes to obtain subjective QoE evaluation and has been used as one of the standardized methods in subjective experiments so far. Zhu etal. [53], however, pointed out QoE measured through MOS
is perceived by an “average user”. Individual preferences
have mostly been neglected, due to the inherent difficulty indealing with quantifying and measuring individual characteristics. Accordingly, they conducted experiments to mimicusers’ viewing experience in real life and reflecting users’
natural behavior by using a Facebook application YouQ described in Sect. 4. They performed the YouQ experiments in
two different sessions. With 136 subjects in the online environment and 20 subjects in the laboratory environment, they

IEICE TRANS. COMMUN., VOL.E104–B, NO.7 JULY 2021

722

concluded that individual QoE evaluations collected through
YouQ were consistent with those reported in the previous literatures [53], [54].
Necessity of the individual QoE, it is underpinned by
the strong assertion “There is no such thing as an “average”
user” by Zhu et al. [3]. Although it has not progressed toindividual adaptation of each person, Huong Pham-Thi etal. [47] classified users into several groups with different
personalities. Consequently, resource allocation by the usergroups with different utility functions outperformed homogeneous user setting, that is a group of average users. Personalization of service must become the next research target
in QoE studies, and individual QoE must be one of the key
points to realize personalized services.
Zhu et al. [53] also identified user factors on QoE: psychological, socio-cultural, demographic, and physiologicalfactors.
1. Physiological Factors are related with the human sensory system. Models of visual perception and attention
have been incorporated in objective video quality assessment metrics.
2. Socio-Cultural Factors are related with the educational
and the socio-cultural background of the user. For example, cultural background has been shown to influence visual perception and attention between Westerners and Asians [55].
3. Demographic Factors (e.g., age, biological sex, or nationality) may influence QoE. For example, the factthat elderly people are found to be more critical than
younger users suggests that elderly people usually havehigher requirements for QoE [56].
4. Psychological Factors are the affective state or theuser’s mood, for instance. Möller and Raake foundthat the user’s mood has been shown to influence quality preferences, and multimedia experiences have beenshown to influence mood in turn [57].
As the physiological factors are firstly listed in the above,
recently, instead of MOS, an advanced approach using physiological measures is being watched with intense interest
to evaluate the video streaming services [58]. Particularly,
electroencephalogram (EEG) is often used to evaluate multimedia quality [58], [59]. Yamazaki [60] also analyzed
the relationship between the quality of the communication
service and the biometric signal in a mobile role-playing
game situation. Subjective QoE assessment data and EEG
data during game operation were collected and the relationship between QoE and EEG has been verified by analysis
of correlation between the QoE evaluation and the EEG
data of gaming. Thus, physiological assessment such as
EEG or SCA is starting to attract attention, because it canevaluate the relationship between physical stimuli and conscious perceptions quantitatively [61]. Since the physiological methodologies measure implicitly the users’ evaluation
for services, they might be able to compensate explicit measurement methodologies such as MOS.

6.

Conclusion

This paper described the present state and future prospect of
QoE studies. Since the end user ultimately decides the service quality, subjective quality evaluation of the user cannot be avoided. Accordingly, collaborative researches on
psychology, physiology, intension and feeling of the humanbeings, etc. must be necessary. Once QoE evaluation is determined and obtained for the Internet service providers or
the network operators, they would control service, application, or network parameters, so that the optimal QoE will
be offered to the end users. Therefore, technologies of themultimedia codecs and / or network engineering should alsocollaborate with QoE studies. Consequently, QoE studies
must be truly interdisciplinary.
On the other hand, different users, different services,and different applications hold different QoE requirements.
As a result, it seems to be difficult to have any common research platform or basement for QoE studies. Nevertheless,since several subclasses of QoE studies have been clarified
through the previous devoted works, one QoE management
architecture was proposed in this paper. The architecture
consists of QoE measurement, QoE assessment, QoS-QoE
mapping, QoE modeling, and QoE adaptation. The categorization of QoE subproblems presented in this paper may
be controversial. Actually, it is true that QoE measurement
and QoE assessment are equated in many studies. Standardization of the types and concepts of QoE factors as well ascomparison of different QoE models are open issues for further study.
Next, development and deployment of video streaming
services based on QoE were surveyed and discussed, because video traffic occupies the network bandwidth mostly
and many users enjoy watching videos anytime and anywhere. It is important for the service providers not to lose
the customers by providing appropriate service quality according to the customers’ QoE evaluation. Although many
researchers and developers in the world focus on this study,
there remain several open challenges and issues such as QoE
modeling, QoE measurement, privacy issues, stakeholder,
and QoE-based management [29].
Finally, novel challenges for QoE studies were introduced as future prospect: one was individual QoE, andthe other was physiology-based QoE measurement. Relating with measurement and collection of individual QoE,
physiology-based QoE measurement might play an important role. As examples of EEG and SCA were introduced,
perceptual changes associated with video quality degradation could be detected by the physiological assessment. Development of the wearable devices with physiological sensing may bring new evolution to QoE studies.
References[1] Q. Dai, “A survey of quality of experience,” R. Lehnert, eds.,
Energy-Aware Communications. EUNICE 2011. Lecture Notes inYAMAZAKI: QUALITY OF EXPERIENCE (QOE) STUDIES: PRESENT STATE AND FUTURE PROSPECT

723

Computer Science, vol.6955, pp.146–156, Springer, Berlin, Heidelberg, 2011.
[2] M. Alreshoodi and J. Woods, “Survey on QoE/QoS correlation models for multimedia services,” International Journal of Distributed andParallel Systems (IJDPS), vol.4, no.3, pp.53–72, May 2013.
[3] Y. Zhu, S.C. Guntuku, W. Lin, G. Ghinea, and J.A. Redi, “Measuring
individual video QoE: A survey, and proposal for future directions
using social media,” ACM Trans. Multimedia Comput. Commun.
Appl., vol.14, no.25, May 2018.
[4] Q. Wang, H.-N. Dai, D. Wu, and H. Xiao, “Data analysis on video
streaming QoE over mobile networks,” EURASIP J. Wireless Com.
Network., vol.2018, no.173, 2018.
[5] Cisco, “Complete forecast update, 2017-2022,” Cisco Visual Network Index, 2018.
[6] L.-C.E. Law, V. Roto, M. Hassenzahl, A.P.O.S. Vermeeren, and J.
Kort, “Understanding, scoping and defining user eXperience: A survey approach,” Proc. 27th International Conference on Human Factors in Computing Systems, CHI 2009, pp.719–728, April 2009.
[7] M. Hassenzahl, “User experience (UX): Towards an experiential
perspective on product quality,” Proc. 20th International Conferenceof the Association Francophone d’Interaction Homme-Machine on,
IHM’08, 2008.
[8] ITU-T Recommendation: E.800, “Definitions of terms related toquality of service - ITU,” Sept. 2008.
[9] S. Karim, H. He, A.A. Laghari, and H. Madiha, “Quality of service(QoS): Measurements of video streaming,” International Journal ofComputer Science Issues (IJCSI), 16, 6, pp.1–8, Nov. 2019.
[10] ITU-T Recommendation P.10/G.100 Amendment 1, “New appendix
I -definition of quality of experience (QoE),” Jan. 2007.
[11] ISO 9241-210:2019, “Ergonomics of human-system interaction Part 210: Human-centred design for interactive systems,” July 2019.
[12] P.L. Callet, S. Möller and A. Perkis, eds., Qualinet White Paper on
Definitions of Quality of Experience, Edition 1.2, EU COST action
1003 QUALINET, March 2013.
[13] ITU-T Recommendation P.10/G.100 (11/17), “Vocabulary for performance, quality of service and quality of experience,” Nov. 2017.
[14] M. Yang, S. Wang, R.N. Calheiros, and F. Yang, “Survey on QoE
assessment approach for network service,” IEEE Access, vol.6,
pp.48374–48390, Aug. 2018.
[15] M. Dong, T. Kimata, K. Sugiura, and K. Zettsu, “End user perceptual
distorted scenes enhancement algorithm using partition-based local
color values for QoE-guaranteed IPTV,” IEICE Trans. Inf. & Syst.,
vol.E97-D, no.10, pp.2606–2612, Oct. 2014.
[16] P. Ameigeiras, J.J. Ramos-Munoz, J. Navarro-Ortiz, P. Mogensen,
and J.M. Lopez-Soler, “QoE oriented cross-layer design of a resource allocation algorithm in beyond 3G systems,” Comput. Commun., vol.33, no.5, pp.571–582, March 2010.
[17] M.A. Khan and U. Toseef, “User utility function as quality of experience (QoE),” Tenth International Conference on Networks (ICN
2011), pp.99–104, Jan. 2011.
[18] D. Schroeder, A.E. Essaili, E. Steinbach, Z. Despotovic, and W.
Kellerer, “A quality-of-experience driven bidding game for uplink
video transmission in next generation mobile networks,” IEEE International Conference on Image Processing (ICIP 2012), USA, Sept.
2012.
[19] K. Piamrat, C. Viho, J.-M. Bonnin, and A. Ksentini, “Quality of experience measurements for video streaming over wireless networks,”
Proc. Sixth International Conference on Information Technology,
vol.1, pp.1184–1189, April 2009.
[20] D. Pal and V. Vanijja, “Effect of network QoS on user QoE for amobile video streaming service using H.265/VP9 codec,” Procedia
Computer Science, vol.111, no.10, pp.214–222, 2017.
[21] ITU-T Recommendation P.800.1, “Mean Opinion Score (MOS) Terminology,” 2003.
[22] ITU-R Recommendation BT.500-11, “Methodology for the subjective assessment of the quality of television pictures,” 2002.
[23] K. Casler, L. Bickel, and E. Hackett, “Separate but equal? A compar-

ison of participants and data gathered via Amazon’s MTurk, social
media, and face-to-face behavioral testing,” Comput. Hum. Behav.,vol.29, no.6, pp.2156–2160, Nov. 2013.
[24] S. Niida, S. Uemura, and H. Nakamura, “Mobile services — User
tolerance for waiting time —,” IEEE Veh. Technol. Mag., vol.5,
no.3, pp.61–67, Sept. 2010.
[25] K. Seshadrinathan, R. Soundararajan, A. Bovik, and L.K. Cormack,“Study of subjective and objective quality assessment of video,”
IEEE Trans. Image Process., vol.19, no.6, pp.1427–1441, June
2010.
[26] T. Yamazaki, R. Yamamoto, and A. Iijima, “Study of relationshipbetween physiological index and quality of experience for video
streaming service,” IEICE Communications Express, vol.7, no.6,
pp.218–223, 2018.
[27] P. Huang and Y. Ishibashi, “QoS control and QoE assessment in
multi-sensory communications with haptics,” IEICE Trans. Commun., vol.E96-B, no.2, pp.392–403, Feb. 2013.
[28] T. Hoßfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold,
and P. Tran-Gia, “Best practices for QoE crowdtesting: QoE assessment with crowdsourcing,” IEEE Trans. Multimedia, vol.16, no.2,
pp.541–558, Feb. 2014.
[29] N. Barman and M.G. Martini, “QoE modeling for HTTP adaptive
video streaming — A survey and open challenges,” IEEE Access,vol.7, no.99, pp.30831–30859, March 2019.
[30] T. Hoßfeld, P.E. Heegaard, M. Varela, L. Skorin-Kapov, and M.
Fiedler, “From QoS distributions to QoE distributions: A system’s
perspective,” Proc. 4th International Workshop on Quality of Experience Management (QoE Management 2020), March 2020.
[31] K. Nahrstedt and R. Steinmetz, “Resource management in multimedia networked systems, handbook of multimedia networking,” J.
Cavanagh, ed., chapter7.3, pp.153–162, Lecture Notes in Computer
Science 1209, Springer Verlag, Sept. 1997.
[32] S. Tasaka and Y. Ishibashi, “A performance comparison of singlestream and multi-stream approaches to live media synchronization,”
IEICE Trans. Commun., vol.E81-B, no.11, pp.1988–1997, Nov.
1998.
[33] S. Tasaka and Y. Ishibashi, “Mutually compensatory property ofmultimedia QoS,” Conf. Rec. IEEE ICC 2002, pp.1105–1111,
April–May 2002.
[34] T. Yamazaki, “Spline-based QoS mapping mechanisms for hierarchical multilevel QoS models,” IEICE Trans. Fundamentals,
vol.E85-A, no.6, pp.1349–1351, June 2002.
[35] K. Fukuda, N. Wakamiya, M. Murata, and H. Miyahara, “Effective algorithms for multicast video transport to meet various QoS
requirements,” IEICE Trans. Commun., vol.E81-B, no.8, pp.1599–
1607, Aug. 1998.
[36] A. Richards, G. Rogers, M. Antoniades, and V. Witana, “Mappinguser level QoS from a single parameter,” Proc. Second IFIP IEEE
Intern. Conf. Management of Multimedia Networks and Services’98 (MMNS’98), Nov. 1998.
[37] A. Benmir, A. Korichi, A. Bourouis, M. Alreshoodi, and L. AlJobouri, “GeoQoE-Vanet: QoE-aware geographic routing protocol
for video streaming over vehicular ad-hoc networks,” Computers,
vol.9, no.2, 45, May 2020.
[38] M. Haklay and P. Weber, “OpenStreetMap: User-generated street
maps,” IEEE Pervasive Comput., vol.7, no.4, pp.12–18, 2008.
[39] P. Georgopoulos, Y. Elkhatib, M. Broadbent, M. Mu, and N.J.P.
Race, “Towards network-wide QoE fairness using OpenFlowassisted adaptive video streaming,” Proc. 2013 ACM SIGCOMM
Workshop on Future Human-Centric Multimedia Networking(FhMN’13), pp.15–20, Aug. 2013.
[40] S. Egger, P. Reichl, T. Hoßfeld, and R. Schatz, “ “Time is Bandwidth”? Narrowing the gap between subjective time perception andquality of experience,” IEEE ICC 2012 - Communication QoS, Reliability and Modeling Symposium (ICC’12 CQRM), June 2012.
[41] V. Menkovski, A. Oredope, A. Liotta, and A. Cuadra Sánchez, “Optimized online learning for QoE prediction,” Proc. 21st Benelux

IEICE TRANS. COMMUN., VOL.E104–B, NO.7 JULY 2021

724

Conference on Artificial Intelligence (BNAIC’09), pp.169–176,
Oct. 2009.
[42] T. Yamazaki, M. Eguchi, T. Miyoshi, and K. Yamori, “A service
quality coordination model bridging QoS and QoE,” 20th International Workshop on Quality of Service (IEEE/ACM IWQoS 2012),
June 2012.
[43] L. Skorin-Kapov and M. Varela, “A multi-dimensional view of QoE:
the ARCU model,” Proc. 35th Int. Conv. MIPRO, pp.662–666, May
2012.
[44] A. Benmir, A. Korichi, A. Bourouis, and M. Alreshoodi, “Survey on
QoE/QoS correlation models for video streaming over vehicular adhoc networks,” J. Computing and Information Technology, vol.26,
no.4, pp.267–287, Dec. 2018.
[45] R. Schatz, S. Schwarzmann, T. Zinner, O. Dobrijevic, E. Liotou, P.
Pocta, S. Barakovic, J.B. Husic, and L. Skorin-Kapov, “QoE management for future networks,” Autonomous Control for a Reliable
Internet of Services: Methods, Models, Approaches, Techniques,
Algorithms, and Tools, I. Ganchev, R.D. van der Mei, and H. vanden Berg, eds., LNCS 10768, pp.49–80, Springer International Publishing, 2018.
[46] D.o Kwon, R. Thay, H. Kim, and H. Ju, “QoE-based adaptive
mVoIP service architecture in SDN networks,” Thirteenth International Conference on Networks (ICN 2014), pp.62–67, Feb. 2014.
[47] H. Pham-Thi, H. Hoang-Van, T. Miyoshi, and T. Yamazaki, “QoEdriven bandwidth allocation method based on user characteristics,”
16th Asia-Pacific Network Operations and Management Symposium(APNOMS 2014), Sept. 2014.
[48] A. Takahashi, D. Hands, and V. Barriac, “Standardization activities
in the ITU for a QoE assessment of IPTV,” IEEE Commun. Mag.,
vol.46, no.2, pp.78–84, March 2008.
[49] ITU-T Recommendation: P.1203, “Parametric bitstream-based quality assessment of progressive download and adaptive audiovisualstreaming services over reliable transport,” Nov. 2016.
[50] K. Yamagishi and T. Hayashi, “Parametric quality-estimation model
for adaptive-bitrate-streaming services,” IEEE Trans. Multimedia,
vol.19, no.7, pp.1545–1557, July 2017.
[51] M.A. Siddik, S.S. Moni, M.S. Alam and W.A. Johnson, “SAFEMAC: Speed aware fairness enabled MAC protocol for vehicularad-hoc networks,” Sensors, vol.19, no.10, May 2019.
[52] U.A. Khan and S.S. Lee, “Multi-layer problems and solutions in
VANETs: A review,” Electronics, vol.8, no.2, 204, Feb. 2019.
[53] Y. Zhu, I. Heynderickx, and J.A. Redi, “Understanding the role ofsocial context and user factors in video quality of experience,” Comput. Hum. Behav., vol.49, pp.412–426, 2015.
[54] M.J. Scott, S.C. Guntuku, Y. Huan, W. Lin, and G. Ghinea, “Modelling human factors in perceptual multimedia quality: On the roleof personality and culture,” Proc. 23rd ACM International Conference on Multimedia (MM’15), pp.481–490, Oct. 2015.
[55] R.E. Nisbett and Y. Miyamoto, “The influence of culture: Holistic
versus analytic perception,” Trends Cogn. Sci., vol.9, no.10, pp.467–
473, 2015.
[56] I. Wechsung, M. Schulz, K.P. Engelbrecht, J. Niemann, and S.
Möller, “All users are (not) equal — The influence of user characteristics on perceived quality, modality choice and performance,” Proc.
Paralinguistic Information and its Integration in Spoken Dialogue
Systems Workshop, Springer, New York, NY, 2011.
[57] S. Möller and A. Raake, Quality of Experience: Advanced Concepts, Applications and Methods, Springer International Publishing
Switzerland, 2014.
[58] S. Arndt, B. Kjell, E. Cheng, U. Engelke, S. Moller, and J.N. Antons,“Review on using physiology in quality of experience,” Electronic
Imaging, vol.2016, no.16, pp.1–9, March 2016.
[59] P. Seeling, “Image quality in augmented binocular vision: QoE approximations with QoS and EEG,” Periodica Polytechnica ElectricalEngineering and Computer Science, vol.61, no.4, pp.327–336, Aug.
2017.
[60] T. Yamazaki, “Case studies on biometric application for quality-of-experience evaluation in communication,” Advances in Biometrics: Modern Methods and Implementation Strategies, G. Sinha, ed.,Springer, pp.227–247, Dec. 2019.
[61] U. Engelke, S. Bosse, M.G. Martini, S. Arndt, D.P. Darcy, G.H.
Mulliken, J.-N. Antons, K.Y. Chan, N. Ramzan, and K. Brunnstr,“Psychophysiology-based QoE assessment: A survey,” IEEE J. Sel.
Topics Signal Process., vol.11, no.1, pp.6–21, Feb. 2017.

Tatsuya Yamazaki
received his B.E., M.E.,
and Ph.D. degrees in information engineeringfrom Niigata University, Japan, in 1987, 1989,
and 2002, respectively. From 1989 to 2013, he
was a researcher at National Institute of Information and Communications Technology (formerly Communications Research Laboratory),
Japan. Since 2013, he has been a professor at
Niigata University. He is also the head of Big
Data Activation Research Center in Niigata University. His research interests are in communication service quality, sensing technologies, image processing, data engineering, machine learning, and pattern recognition. He is a member of theIEEE, the Information Processing Society of Japan, the Institute of Image
Information and Television Engineers, the Japanese Society for Artificial
Intelligence, and Human Interface Society.