Patent Description:
Inference of QoE of one or more users, and the detection of its degradation, is an important management tool for internet service providers (ISPs). Many objective in-browser metrics exist, and their monitoring from encrypted web traffic between the users and web applications is feasible. Thereby, the web traffic corresponds to the amount of data sent or received by the users to/from the web applications.

However, QoE degradations may be rooted at different sources. For instance, QoE degradations may be rooted in network-induced problems, i.e., problems related to the network that impacts the web traffic and thus causes the QoE degradations. Alternatively, the QoE degradations could be simply caused by a change of the content itself.

Some relevant studies in this field relate to the definition of QoE and its inference. In particular, some conventional techniques define web QoE metrics, which correlate with the QoE performance, so as to infer the web QoE metrics without having to access browser metrics. The QoE inference may be done from encrypted network traffic, which is what ISPs typically have access to.

Some other studies relate to web QoE monitoring. For instance, browser studies typically correlate a QoE change with changes in the software base.

Some other studies relate more specifically to QoE degradation, in particular, the detection of QoE degradations of users. The focus on QoE degradations from the perspective of the ISPs was proposed first for tackling general applications, then video applications, and more recently also web applications.

Some other studies relate to web content monitoring. Given that the web evolved much in the last thirty years, there is industrial effort to monitor its evolution, in particular, from the viewpoint of web page characteristics, or technologies, or content. Academia also regularly performs longitudinal surveys of content changes employing large datasets, but QoE is generally out of the picture in these studies.

<CIT> discloses a QoE optimization system analyzes diagnostic files to determine device Key Performance Indicators and a device QoE, and to determine a root cause of a network problem (such as dropped calls) leading to a diminished QoE. <CIT> discloses that the single QoE metric may be determined based on a set of empirical characteristics relating to the streaming video service such as startup latency, video quality, and the likelihood of interruptions in streaming playback.

The present disclosure bases on the fact that the conventional studies focus either on QoE or on content, but rarely are both aspects considered together. Furthermore, even when these aspects are considered together, only marginals are considered, but temporal aspects are not taken into account. That is, no information from content evolution is taken into account when evaluating QoE.

QoE degradations are conventionally identified, for instance, by linking network key performance indicator (KPI) features with application-specific metrics representative of user QoE (also called user KQI). However, this neglects the impact of content changes that regularly happen in web applications. For instance, when a web page of a web application changes, the KQI indicator can change as well (e.g., when the structure of the web page becomes more complex with more objects, it may take longer to download the web page, but this may not be related to any network limitation or to a reduced web traffic). The conventional methodology, which ignores content particularly content changes, is thus not reliable in the case of web applications, and would lead to many false alarms.

In view of the above-mentioned problems and disadvantages, embodiments of the present invention aim to improve the detection of QoE degradations. In particular, an objective is to enable a joint monitoring of QoE and web application content, in order to determine the network-induced contribution to a QoE degradation of a user, or of a set of users, of the web application.

The objective is achieved by the embodiments of the invention as described in the enclosed independent claims. Advantageous implementations of the embodiments of the invention are further defined in the dependent claims.

Embodiments of the invention provide a device and a method for detecting QoE degradations of users of web applications, wherein, instead of looking at the network level only, also content related information is considered. A device or method according to an embodiment of the invention may, to this end, employ active measurements for accessing the content related information. The embodiments of the invention base on solving two problems. Firstly, the problem of identifying and disentangling QoE degradations caused by network issues (for which ISPs are responsible) from apparent QoE degradations caused by content-based changes (for which ISPs are not responsible). Secondly, the problem of combining different network and content signals, in order to provide clues about the most probable origin of the determined QoE degradations.

A first aspect of the disclosure provides a device for detecting a network-induced contribution to a QoE degradation for one or more web applications, as defined by the appended independent claim <NUM>.

The device of the first aspect is able to consider the content particularly content changes of the web applications, when detecting a QoE degradation of a user. For instance, the device may passively monitor the one or more user metrics (which may comprise one or more quality of service (QoS) metrics), and may further separately perform active measurements to track changes of the content for one or more selected web applications, i.e., to determine the content metrics.

The device may detect the QoE degradation following the user metrics. The one or more user metrics may be closely related to the QoE of the one or more users. The user metrics may be relevant to the one or more users (instead of purely network related metrics), and may be accessible (passively) even under encryption of the web traffic.

The device can further determine the network-induced QoE contribution to the QoE degradation. For instance, the device may distinguish, whether the determined QoE degradation is caused by a network-based issue or change and/or is caused by a content-based change. This can improve the detection and analysis of the determined QoE degradation. In particular, this may enable a reduction of false alarms. It may further be possible for the device to explain the determined QoE degradation, e.g., to determine the contribution to the QoE degradation of either content, network, or another source. This may allow for a better understanding of issues encountered by the users.

In an implementation form of the first aspect, the contribution information is indicative of one of: the network-induced contribution is the largest contribution to the determined QoE degradation; the network-induced contribution is not the largest contribution to the determined QoE degradation; the network-induced contribution is the only contribution to the determined QoE degradation; the network-induced contribution to the determined QoE degradation is zero.

The network-induced contribution is the contribution that the network (particularly, a network issue) has on the determined QoE degradation. The determined QoE degradation reflects the change of the QoE (and corresponding change of the user metrics) in a certain direction considered to indicate a worse QoE. That is, the network-induced contribution is the contribution, which may be in the responsibility of the ISP.

According to this implementation form, the network-induced contribution may be <NUM>% (i.e., the determined QoE degradation is only caused by network issue(s)), or may be <NUM>% (i.e., there is no contribution of the network to the QoE degradation). A distinction may be further made, whether the network-induced contribution is above <NUM>% of the total contribution to the determined QoE degradation, or equal to or less than <NUM>%. The network-induced contribution may, however, be the largest contribution, even if its contribution is less than <NUM>% (e.g., among a contribution if network, content, or another cause).

In an implementation form of the first aspect, the device is further configured to: obtain the content metrics by measuring the content of the web applications.

In particular, a change of the content of the one or more web applications (over time) may be obtained by the device. The device may perform active measurements to obtain the one or more content metrics, for instance, from one or more web pages of the web applications.

In an implementation form of the first aspect, the device is further configured to: obtain the user metrics by monitoring network traffic.

In particular, a change of the one or more user metrics (over time) may be obtained by the device, for instance, may be collected by the device. The device may perform passive measurements to obtain these user metrics, for instance, from encrypted or unencrypted web traffic.

In an implementation form of the first aspect, the device is further configured to, if the contribution information is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation: obtain one or more network metrics, the network metrics being related to a performance of the network; and determine whether a change of at least one of the network metrics corresponds to the QoE degradation.

In particular, a change of the one or more network metrics (over time) may be obtained by the device, for instance, may be collected by the device. The device may perform passive and/or active measurements to obtain these network metrics, for instance, from other network elements.

In an implementation form of the first aspect, the device is further configured to: report the at least one of the network metrics corresponding to the QoE degradation; and/or rank the at least one of the network metrics corresponding to the QoE degradation according to a predefined criterion.

In an implementation form of the first aspect, the device is further configured to: report an unidentified network issue, if there is no change of at least one of the network metrics corresponding to the QoE degradation.

In an implementation form of the first aspect, the device is further configured to: estimate a value of the network-induced contribution to the QoE degradation; and/or estimate a value of a content-induced contribution to the QoE degradation, wherein the content-induced contribution is caused by a content change of the content of the web applications.

In an implementation form of the first aspect, the device is further configured to: obtain one or more raw user metrics and/or one or more raw content metrics; and obtain the user metrics and/or the content metrics by performing a change point detection (CPD) algorithm and/or an outlier detection (OD) algorithm on, respectively, the raw user metrics and/or the raw content metrics.

A raw user metric may be a time series of measurements of a user metric over one user or over a pool of users of the one or more web applications. A raw content metric may be a time series of measurements of a content metric over one machine or over a pool of machines. One or more processing operations may be carried out on the respective raw metric to obtain the corresponding metric, for instance, processing operations comprising CPD and OD operations.

In an implementation form of the first aspect, the user metrics comprise at least one of: one or more QoS parameters related to the QoE of the users; one or more QoE indicators indicative of the QoE of the users; one or more key quality indicators related to the QoE of the users.

The QoS parameters may be related to one or more web applications.

In an implementation form of the first aspect, the user metrics comprise at least one of: a page loading time of one or more web pages related to the web applications; a speed index of one or more web pages related to the web applications; a byte index of one or more web pages related to the web applications.

In an implementation form of the first aspect, the content metrics comprise at least one of:
a number of objects of one or more web pages related to the web applications; a page weight of one or more web pages related to the web applications.

A second aspect of the disclosure provides a method for detecting a network-induced contribution to a QoE degradation for one or more web applications, as defined by the appended independent claim <NUM>.

A third aspect of the disclosure, as defined by the appended independent claim <NUM>, provides a computer program comprising a program code for performing the method according to the second aspect when executed on a computer.

<FIG> shows a device <NUM> according to an embodiment of the invention. The device <NUM> is configured to detect QoE degradation(s) of one or more users of one or more web applications. In particular, the device <NUM> is configured to detect a network-induced contribution to the detected QoE degradation(s), i.e., to determine, to which extent the network (particularly a network issue) is responsible for the QoE degradation(s). The device may be any computer or processing device, which is configured to perform the steps described in the following.

The device <NUM> is configured to obtain one or more user metrics <NUM>, wherein the one or more user metrics <NUM> are indicative of a QoE of one or more users of the one or more web applications. The device <NUM> is not limited by how it obtains these user metrics <NUM>. For instance, the device <NUM> may monitor network traffic (particularly web traffic related to the one or more web applications), in order to obtain the one or more user metrics <NUM>. The network traffic may be encrypted or non-encrypted. The device <NUM> may collect the user metrics <NUM> from the network traffic.

Further, the device <NUM> is configured to obtain one or more content metrics <NUM>, wherein the one or more content metrics <NUM> are indicative of a content of the one or more web applications. The device <NUM> is not limited by how it obtains these content metrics <NUM>. For instance, the device <NUM> may measure the content of the one or more web applications, in order to obtain the one or more content metrics <NUM>. For instance, the device <NUM> may measure a number of objects of one or more web pages related to the one or more web applications, and/or a page weight of one or more web pages related to the one or more web applications, in order to obtain the one or more content metrics <NUM>. Thereby, the one or more content metrics <NUM> may comprise the number of objects and/or may comprise the page weight.

Further the device <NUM> is configured to determine a QoE degradation <NUM> of the QoE of the one or more users, based on the one or more user metrics <NUM>. For instance, the device <NUM> may detect a change in the one or more user metrics <NUM>, and may determine that the change is associated with, or indicates, a degradation of the QoE of the users associated with the one or more user metrics <NUM>. The QoE degradation <NUM> of the one or more users may be temporally correlated with the change (e.g., a degradation) of the one or more user metrics <NUM>.

The device <NUM> is then configured to determine a contribution information <NUM>, which is indicative of a network-induced contribution to the determined QoE degradation <NUM>. The device <NUM> is configured to determine the contribution information <NUM> based on the one or more content metrics <NUM>. The contribution information <NUM> may be indicative of one of: the network-induced contribution is the largest contribution to the determined QoE degradation <NUM>, e.g., it contributes by more than <NUM>%; the network-induced contribution is not the largest contribution to the determined QoE degradation <NUM>, e.g., it contributes by less than <NUM>%; the network-induced contribution is the only contribution to the determined QoE degradation <NUM>, i.e., it contributes completely by <NUM>%; or the network-induced contribution to the determined QoE degradation <NUM> is zero, i.e., it contributes <NUM>%.

The device <NUM> may comprise a processor or processing circuitry (not shown) configured to perform, conduct or initiate the various operations of the device <NUM> described herein. The processing circuitry may comprise hardware and/or the processing circuitry may be controlled by software. The hardware may comprise analog circuitry or digital circuitry, or both analog and digital circuitry. The digital circuitry may comprise components such as application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), digital signal processors (DSPs), or multipurpose processors.

The device <NUM> may further comprise memory circuitry, which stores one or more instruction(s) that can be executed by the processor or by the processing circuitry, in particular under control of the software. For instance, the memory circuitry may comprise a non-transitory storage medium storing executable software code which, when executed by the processor or the processing circuitry, causes the various operations of the device <NUM> to be performed.

In one embodiment, the processing circuitry comprises one or more processors and a non-transitory memory connected to the one or more processors. The non-transitory memory may carry executable program code which, when executed by the one or more processors, causes the device <NUM> to perform, conduct or initiate the operations or methods described herein.

According to the above description, the device <NUM>, according to embodiments of the invention, may identify a QoE degradation <NUM> of one or more users to be specifically caused by a network issue (i.e., identify that it is network-induced), either partly or completely. For instance, the device <NUM> may use active web measurements to disentangle apparent issues, which are due to changes in the content. A possible device <NUM> synoptic will be described in the following with respect to <FIG>.

<FIG> shows a device <NUM> according to an embodiment of the invention, which builds on the embodiment shown in <FIG>. Same elements in <FIG> and <FIG> are labelled with the same reference signs and may be implemented likewise. <FIG> shows, in particular, a possible synoptic of the device <NUM>.

The device <NUM> may obtain input as follows:.

The device <NUM> may further process the input as follows:.

<FIG> shows schematically a method carried out by a device <NUM> according to an embodiment of the invention. The device <NUM> may be as shown in <FIG> or <FIG>. Same elements in <FIG> and <FIG> and <FIG> are labelled with the same reference signs and may be implemented likewise.

The method performed by the device <NUM> may extract the most probable "cause" of a QoE degradation <NUM> among the possible contributions including {content, network, unknown}. The method may comprise comparing one or more time series of user events with content events, as reflected in the one or more user metrics <NUM> and the one or more content metrics <NUM>, respectively. An underlying idea of the method performed by the device <NUM> is explained with respect to <FIG> in a simple high-level heuristic following a tree model for simplicity, wherein binary logic may construct a full truth table.

The device <NUM> may be configured to track QoE metrics (i.e., a user metric <NUM>) of a user or a pool of users of a web application. The user metric <NUM> may, for example, comprise a PLT, and is represented by "U" in <FIG>.

The device <NUM> may determine a QoE degradation <NUM> based on the user metric <NUM>. When a QoE degradation <NUM> is not observed, the device <NUM> may determine that there is no QoE issue, and may provide an according output <NUM>. When the device <NUM> determines a QoE degradation <NUM> for this user or pool of users, the device <NUM> may continue by checking the content of the web application.

In particular, the device <NUM> may track a content metric <NUM>, for example, the number of objects of a web page of the web application, represented by "C" in <FIG>. When the device <NUM> determines, based on the content metric <NUM>, that a change of the content corresponds to - for instance, is concomitant - with the QoE degradation <NUM> determined based on the user metric <NUM>, the device <NUM> may determine that there is no QoE issue, and that instead the QoE degradation is due to a content change of the web application. The device <NUM> may provide an according output <NUM>. When the device <NUM> determines that a change of the content is not observed concomitantly, the device may raise a network issue alarm. Further, the device <NUM> may provide the contribution information <NUM>, which is indicative of the network-induced contribution to the determined QoE degradation <NUM>.

The QoE degradation <NUM> may correspond to a specific instant given by an algorithm, typically a CPD algorithm, over a univariate or multivariate time series. The term "concomitant" may correspond to either occurring around the same time (with around defined by a certain margin of error in time the device <NUM> allows), or a more complex algorithm for detecting concomitance.

<FIG> shows schematically a method carried out by a device <NUM> according to an embodiment of the invention, which builds on <FIG>. The device <NUM> may be as shown in <FIG> or <FIG>. Same elements in <FIG> and <FIG> are labelled with the same reference signs and may be implemented likewise.

<FIG> particularly shows that the device <NUM> can further refine the identification of the network issue, for instance, by tracking down network QoS metrics over the network. When the device <NUM> raises a network issue, the device <NUM> may be further configured to check a network metric <NUM> (represented by "N" in <FIG>) tracked over the network. For instance, if the contribution information <NUM> is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation <NUM>, the device <NUM> may obtain the network metric <NUM>, wherein the network metric <NUM> is related to a performance of the network. Further, the device <NUM> may determine whether a change of the network metric <NUM> corresponds to the QoE degradation <NUM>. If a change of the network metric <NUM> corresponds to the QoE degradation <NUM> - for instance they are concomitant - the device <NUM> may provide an output <NUM> indicating the problematic network metric <NUM>. The device <NUM> may in this way also check multiple network metrics <NUM>, and may rank problematic network metrics <NUM> by importance. Otherwise, the device <NUM> may provide an output <NUM> indicating an unidentified network issue.

<FIG> shows a method <NUM> according to an embodiment of the invention. The method <NUM> may be performed by the device <NUM> and is for detecting a network-induced contribution to a QoE, degradation for one or more web applications. The method <NUM> comprises a step <NUM> of obtaining one or more user metrics <NUM>, the user metrics <NUM> being indicative of a QoE of one or more users of the web applications. The method <NUM> further comprises a step <NUM> of obtaining one or more content metrics <NUM>, the content metrics <NUM> being indicative of a content of the web applications. The method <NUM> further comprises a step <NUM> of determining a QoE degradation of the QoE of the users based on the one or more user metrics <NUM>. Finally, the method <NUM> comprises a step <NUM> of determining a contribution information indicative of a network-induced contribution to the determined QoE degradation <NUM> based on the content metrics <NUM>.

Claim 1:
A device (<NUM>) for detecting a network-induced contribution to a quality of experience, QoE, degradation for one or more web applications, the device (<NUM>) being configured to:
obtain one or more user metrics (<NUM>), the user metrics (<NUM>) being indicative of a QoE of one or more users of the web applications;
obtain one or more content metrics (<NUM>), the content metrics (<NUM>) being indicative of a content of the web applications;
determine a QoE degradation (<NUM>) of the QoE of the users based on the one or more user metrics (<NUM>); and
determine a contribution information (<NUM>) indicative of a network-induced contribution to the determined QoE degradation (<NUM>) based on the content metrics (<NUM>);
wherein the device (<NUM>) is further configured to:
determine whether the QoE degradation (<NUM>) corresponds to a change of the content metrics (<NUM>); and
determine the contribution information (<NUM>), wherein the contribution information (<NUM>) is indicative of the network-induced contribution being the largest contribution to the determined QoE degradation (<NUM>), if the QoE degradation (<NUM>) does not correspond to a change of the content metrics (<NUM>), or
determine the contribution information (<NUM>), wherein the contribution information (<NUM>) is indicative of the network-induced contribution being not the largest contribution to the determined QoE degradation (<NUM>), if the QoE degradation (<NUM>) corresponds to a change of the content metrics (<NUM>).