# Defining and Assessing Quality in IoT Environments: A Survey
## 3 Quality in a IoT Environment  
In telecommunications, the most suitable metric to assess end-to-end quality is QoE. The most frequently used definition for QoE is the one given by the ITU [26], where it is defined as \"The overall acceptability of an application or service, as perceived subjectively by the end-user\". Moreover, since many researchers pointed out that the inclusion of the term \"acceptability\" as the basis for a QoE definition is not ideal, during the Dagstuhl Seminar in 2009, the term acceptability was newly defined as \"the outcome of a decision [yes/no] which is partially based on the Quality of Experience\" [27]. However, even with this modification, the definition still follows a user-centric approach; thus, it does not reflect the machines' perspective.  
Another popular definition of QoE is the one described in the Qualinet White paper [28] in which QoE is defined as \"the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and/or enjoyment of the application or service in the light of the user's personality and current state\". [PERSON] and [PERSON] [27] extended the definition of the Qualinet White paper in order to also include the term system. Thus, according to the new definition QoE is \"the degree of delight or annoyance of a person whose experiencing involves an application, service, or system. It results from the person's evaluation of the fulfillment of his or her expectations and needs with respect to the utility and/or enjoyment in the light of the person's context, personality and current state\".  
The Qualinet's definition according to [PERSON] and [PERSON] [20] is valid for general multimedia applications/services and, thus, it can be used for cases where humans are the recipients of the content provided by multimedia IoT applications.  
However, in the IoT context, where exist applications that do not require any human intervention, such as smart parking, connected vehicles, etc., the term QoE cannot be used to describe quality. To this end, several researchers introduced new terms to define quality in the IoT domain. [PERSON] et al. [29] introduced the term Quality of IoT-experience (QoIoT), which aggregates the delivered quality of an IoT service from the perspective of both humans and machines within the context of autonomous vehicles. More specifically, the QoIoT metric compromises the traditional user-centric QoE metric and the Quality of Machine Experience (QoME), an objective metric that \"measures the quality and performance of intelligent machines and their decisions\".  
[PERSON] et al. [30] defined the term \"Quality of Things\" (QoT) for multimedia communications in IoT to express the quality of fulfilling an IoT task in a Multimedia IoT (M-IoT) [31]. However, the authors do not provide any measurement methodology.  
[PERSON] et al. [9] defined the term Quality of Systems (QoSys), an objective metric like QoE, that measures \"the quality and performance of the Systems of Systems (SoS), and the decisions made by those\". Thus, the metric \(QoE_{IOT}\) is introduced in order to evaluate the quality in an IoT scenario from the perspective of both humans and machines.  
[PERSON] et al. [32] introduced the term quality of X (QoX), as a comprehensive evaluation metric that combines QoS, QoE, Quality of Data (QoD) and Quality of Information (QoI).  
In this end, [PERSON] et al. [17] introduce the term Quality of autonomic IoT applications as \"an aggregate quantitative value of various IoT quality metrics measured at each stage of the autonomic IoT application life cycle\".  
Table 1 overviews and highlights the specific drawbacks of each definition. As can be seen, there is no definition that can generally express the end-to-end quality in IoT environments.