Using throughput mode distribution as a proxy for quality of experience and path selection in the internet

In one embodiment, a device calculates one or more distributions of bitrates associated with an application whose traffic is conveyed via one or more paths in a network. The device detects throughput modes of the application, based on the one or more distributions of bitrates associated with the application. The device associates each throughput mode with a quality of experience label, to form a plurality of pairs of throughput modes and quality of experience labels. The device estimates a quality of experience metric for the application, based on a bitrate of the application and the plurality of pairs of throughput modes and quality of experience labels.

TECHNICAL FIELD

The present disclosure relates generally to computer networks, and, more particularly, to using throughput mode distribution as a proxy for quality of experience and path selection in the Internet.

BACKGROUND

Software-defined wide area networks (SD-WANs) represent the application of software-defined networking (SDN) principles to WAN connections, such as connections to cellular networks, the Internet, and Multiprotocol Label Switching (MPLS) networks. The power of SD-WAN is the ability to provide consistent service level agreement (SLA) for important application traffic transparently across various underlying tunnels of varying transport quality and allow for seamless tunnel selection based on tunnel performance characteristics that can match application SLAs and satisfy the quality of service (QoS) requirements of the traffic (e.g., in terms of delay, jitter, packet loss, etc.).

On the application side, the networking requirements are often poorly understood, even for very common applications such as voice and video for which a variety of metrics have been developed over the past two decades, with the hope of accurately representing the Quality of Experience (QoE) from the standpoint of the users of the application. Typically, QoE is measured by applying static SLA templates to QoS probes. However, this approach often ranges range from being only weakly correlated with the QoE of the application to being completely useless in discerning the QoE of the application.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Overview

According to one or more embodiments of the disclosure, a device calculates one or more distributions of bitrates associated with an application whose traffic is conveyed via one or more paths in a network. The device detects throughput modes of the application, based on the one or more distributions of bitrates associated with the application. The device associates each throughput mode with a quality of experience label, to form a plurality of pairs of throughput modes and quality of experience labels. The device estimates a quality of experience metric for the application, based on a bitrate of the application and the plurality of pairs of throughput modes and quality of experience labels.

DESCRIPTION

2.) Site Type B: a site connected to the network by the CE router via two primary links (e.g., from different Service Providers), with potentially a backup link (e.g., a 3G/4G/5G/LTE connection). A site of type B may itself be of different types:

2a.) Site Type B1: a site connected to the network using two MPLS VPN links (e.g., from different Service Providers), with potentially a backup link (e.g., a 3G/4G/5G/LTE connection).

2c.) Site Type B3: a site connected to the network using two links connected to the public Internet, with potentially a backup link (e.g., a 3G/4G/5G/LTE connection).

According to various embodiments, a software-defined WAN (SD-WAN) may be used in network100to connect local network160, local network162, and data center/cloud environment150. In general, an SD-WAN uses a software defined networking (SDN)-based approach to instantiate tunnels on top of the physical network and control routing decisions, accordingly. For example, as noted above, one tunnel may connect router CE-2 at the edge of local network160to router CE-1 at the edge of data center/cloud environment150over an MPLS or Internet-based service provider network in backbone130. Similarly, a second tunnel may also connect these routers over a 4G/5G/LTE cellular service provider network. SD-WAN techniques allow the WAN functions to be virtualized, essentially forming a virtual connection between local network160and data center/cloud environment150on top of the various underlying connections. Another feature of SD-WAN is centralized management by a supervisory service that can monitor and adjust the various connections, as needed.

In general, routing process (services)244contains computer executable instructions executed by the processor220to perform functions provided by one or more routing protocols. These functions may, on capable devices, be configured to manage a routing/forwarding table (a data structure245) containing, e.g., data used to make routing/forwarding decisions. In various cases, connectivity may be discovered and known, prior to computing routes to any destination in the network, e.g., link state routing such as Open Shortest Path First (OSPF), or Intermediate-System-to-Intermediate-System (ISIS), or Optimized Link State Routing (OLSR). For instance, paths may be computed using a shortest path first (SPF) or constrained shortest path first (CSPF) approach. Conversely, neighbors may first be discovered (e.g., a priori knowledge of network topology is not known) and, in response to a needed route to a destination, send a route request into the network to determine which neighboring node may be used to reach the desired destination. Example protocols that take this approach include Ad-hoc On-demand Distance Vector (AODV), Dynamic Source Routing (DSR), DYnamic MANET On-demand Routing (DYMO), etc. Notably, on devices not capable or configured to store routing entries, routing process244may consist solely of providing mechanisms necessary for source routing techniques. That is, for source routing, other devices in the network can tell the less capable devices exactly where to send the packets, and the less capable devices simply forward the packets as directed.

In various embodiments, routing process244and/or QoE estimation process248may employ one or more supervised, unsupervised, or semi-supervised machine learning models. Generally, supervised learning entails the use of a training set of data, as noted above, that is used to train the model to apply labels to the input data. For example, the training data may include sample telemetry that has been labeled as being indicative of an acceptable QoE or an unacceptable QoE. On the other end of the spectrum are unsupervised techniques that do not require a training set of labels. Notably, while a supervised learning model may look for previously seen patterns that have been labeled as such, an unsupervised model may instead look to whether there are sudden changes or patterns in the behavior of the metrics. Semi-supervised learning models take a middle ground approach that uses a greatly reduced set of labeled training data.

The performance of a machine learning model can be evaluated in a number of ways based on the number of true positives, false positives, true negatives, and/or false negatives of the model. For example, the false positives of the model may refer to the number of times the model incorrectly predicted the QoE of an application not being acceptable. Conversely, the false negatives of the model may refer to the number of times the model incorrectly predicted that the QoE of the application is acceptable. True negatives and positives may refer to the number of times the model correctly predicted an acceptable QoE or an unacceptable QoE, respectively. Related to these measurements are the concepts of recall and precision. Generally, recall refers to the ratio of true positives to the sum of true positives and false negatives, which quantifies the sensitivity of the model. Similarly, precision refers to the ratio of true positives the sum of true and false positives.

As noted above, in software defined WANs (SD-WANs), traffic between individual sites are sent over tunnels. The tunnels are configured to use different switching fabrics, such as MPLS, Internet, 4G or 5G, etc. Often, the different switching fabrics provide different quality of service (QoS) at varied costs. For example, an MPLS fabric typically provides high QoS when compared to the Internet, but is also more expensive than traditional Internet. Some applications requiring high QoS (e.g., video conferencing, voice calls, etc.) are traditionally sent over the more costly fabrics (e.g., MPLS), while applications not needing strong guarantees are sent over cheaper fabrics, such as the Internet.

Traditionally, network policies map individual applications to Service Level Agreements (SLAs), which define the satisfactory performance metric(s) for an application, such as loss, latency, or jitter. Similarly, a tunnel is also mapped to the type of SLA that is satisfies, based on the switching fabric that it uses. During runtime, the SD-WAN edge router then maps the application traffic to an appropriate tunnel. Currently, the mapping of SLAs between applications and tunnels is performed manually by an expert, based on their experiences and/or reports on the prior performances of the applications and tunnels.

The emergence of infrastructure as a service (IaaS) and software as a service (SaaS) is having a dramatic impact of the overall Internet due to the extreme virtualization of services and shift of traffic load in many large enterprises. Consequently, a branch office or a campus can trigger massive loads on the network.

FIGS. 3A-3Billustrate example network deployments300,310, respectively. As shown, a router110(e.g., a device200) located at the edge of a remote site302may provide connectivity between a local area network (LAN) of the remote site302and one or more cloud-based, SaaS providers308. For example, in the case of an SD-WAN, router110may provide connectivity to SaaS provider(s)308via tunnels across any number of networks306. This allows clients located in the LAN of remote site302to access cloud applications (e.g., Office365™, Dropbox™, etc.) served by SaaS provider(s)308.

As would be appreciated, SD-WANs allow for the use of a variety of different pathways between an edge device and an SaaS provider. For example, as shown in example network deployment300inFIG. 3A, router110may utilize two Direct Internet Access (DIA) connections to connect with SaaS provider(s)308. More specifically, a first interface of router110(e.g., a network interface210, described previously), Int 1, may establish a first communication path (e.g., a tunnel) with SaaS provider(s)308via a first Internet Service Provider (ISP)306a, denoted ISP 1 inFIG. 3A. Likewise, a second interface of router110, Int 2, may establish a backhaul path with SaaS provider(s)308via a second ISP306b, denoted ISP 2 inFIG. 3A.

FIG. 3Billustrates another example network deployment310in which Int 1 of router110at the edge of remote site302establishes a first path to SaaS provider(s)308via ISP 1 and Int 2 establishes a second path to SaaS provider(s)308via a second ISP306b. In contrast to the example inFIG. 3A, Int 3 of router110may establish a third path to SaaS provider(s)308via a private corporate network306c(e.g., an MPLS network) to a private data center or regional hub304which, in turn, provides connectivity to SaaS provider(s)308via another network, such as a third ISP306d.

Regardless of the specific connectivity configuration for the network, a variety of access technologies may be used (e.g., ADSL, 4G, 5G, etc.) in all cases, as well as various networking technologies (e.g., public Internet, MPLS (with or without strict SLA), etc.) to connect the LAN of remote site302to SaaS provider(s)308. Other deployments scenarios are also possible, such as using Colo, accessing SaaS provider(s)308via Zscaler or Umbrella services, and the like.

FIG. 4illustrates an example SDN implementation400, according to various embodiments. As shown, there may be a LAN core402at a particular location, such as remote site302shown previously inFIGS. 3A-3B. Connected to LAN core402may be one or more routers that form an SD-WAN service point406which provides connectivity between LAN core402and SD-WAN fabric404. For instance, SD-WAN service point406may comprise routers110a-110b.

Overseeing the operations of routers110a-110bin SD-WAN service point406and SD-WAN fabric404may be an SDN controller408. In general, SDN controller408may comprise one or more devices (e.g., devices200) configured to provide a supervisory service, typically hosted in the cloud, to SD-WAN service point406and SD-WAN fabric404. For instance, SDN controller408may be responsible for monitoring the operations thereof, promulgating policies (e.g., security policies, etc.), installing or adjusting IPsec routes/tunnels between LAN core402and remote destinations such as regional hub304and/or SaaS provider(s)308inFIGS. 3A-3B, and the like.

As noted above, a primary networking goal may be to design and optimize the network to satisfy the requirements of the applications that it supports. So far, though, the two worlds of “applications” and “networking” have been fairly siloed. More specifically, the network is usually designed in order to provide the best SLA in terms of performance and reliability, often supporting a variety of Class of Service (CoS), but unfortunately without a deep understanding of the actual application requirements. On the application side, the networking requirements are often poorly understood even for very common applications such as voice and video for which a variety of metrics have been developed over the past two decades, with the hope of accurately representing the Quality of Experience (QoE) from the standpoint of the users of the application.

More and more applications are moving to the cloud and many do so by leveraging an SaaS model. Consequently, the number of applications that became network-centric has grown approximately exponentially with the raise of SaaS applications, such as Office365, ServiceNow, SAP, voice, and video, to mention a few. All of these applications rely heavily on private networks and the Internet, bringing their own level of dynamicity with adaptive and fast changing workloads. On the network side, SD-WAN provides a high degree of flexibility allowing for efficient configuration management using SDN controllers with the ability to benefit from a plethora of transport access (e.g., MPLS, Internet with supporting multiple CoS, LTE, satellite links, etc.), multiple classes of service and policies to reach private and public networks via multi-cloud SaaS.

Application aware routing usually refers to the ability to route traffic so as to satisfy the requirements of the application, as opposed to exclusively relying on the (constrained) shortest path to reach a destination IP address. Various attempts have been made to extend the notion of routing, CSPF, link state routing protocols (ISIS, OSPF, etc.) using various metrics (e.g., Multi-topology Routing) where each metric would reflect a different path attribute (e.g., delay, loss, latency, etc.), but each time with a static metric. At best, current approaches rely on SLA templates specifying the application requirements so as for a given path (e.g., a tunnel) to be “eligible” to carry traffic for the application. In turn, application SLAs are checked using regular probing. Other solutions compute a metric reflecting a particular network characteristic (e.g., delay, throughput, etc.) and then selecting the supposed ‘best path,’ according to the metric.

As noted above, determining the true QoE of an application is absolutely key in making routing decisions in SDNs, such as SD-WAN networks. However, the current approach to determining the QoE of a cloud-hosted application makes use of static SLA templates applied to QoS probes that often range from being weakly correlated with the true QoE, at best, to being completely useless in discerning the true QoE of the application.

Using Throughput Mode Distribution as a Proxy for QoE and Path Selection

The techniques introduced herein adopt a completely different QoE estimation approach that consists in using traffic telemetry information (e.g., coming from NetFlow, etc.), to infer the QoE for various applications. In some aspects, the techniques herein identify the typical ‘modes’ of throughput for various applications on the network from which their QoE metrics can be inferred. Doing so allows for the QoE metrics to be computed without having to rely on active QoS probing and comparisons to static SLA templates.

Illustratively, the techniques described herein may be performed by hardware, software, and/or firmware, such as in accordance with QoE estimation process248, which may include computer executable instructions executed by the processor220(or independent processor of interfaces210) to perform functions relating to the techniques described herein (e.g., in conjunction with routing process244).

Specifically, according to various embodiments, a device calculates one or more distributions of bitrates associated with an application whose traffic is conveyed via one is or more paths in a network. The device detects throughput modes of the application, based on the one or more distributions of bitrates associated with the application. The device associates each throughput mode with a quality of experience label, to form a plurality of pairs of throughput modes and quality of experience labels. The device estimates a quality of experience metric for the application, based on a bitrate of the application and the plurality of pairs of throughput modes and quality of experience labels.

Operationally,FIG. 6illustrates an example architecture600for estimating the Quality of Experience (QoE) of an application, according to various embodiments. At the core of architecture600is QoE estimation process248, which may be executed by a supervisory device of a network or another device in communication therewith. For instance, QoE estimation process248may be executed by an SDN controller (e.g., SDN controller408inFIG. 4), a particular networking device in the network (e.g., a router, etc.), or another device in communication therewith. As shown, QoE estimation process248may include any or all of the following components: an application bitrate monitor (ABM)502, a throughput mode detector (TMD)504, a QoE inference engine (QIE)506, and/or a QoE validation engine (QVE)508. As would be appreciated, the functionalities of these components may be combined or omitted, as desired. In addition, these components may be implemented on a singular device or in a distributed manner, in which case the combination of executing devices can be viewed as their own singular device for purposes of executing QoE estimation process248.

In various embodiments, application bitrate monitor (ABM)502may receive and analyze telemetry data512from any number of traffic telemetry collectors510in the network under scrutiny. For instance, telemetry data512may comprise NetFlow records, IPFIX records, or the like, that include telemetry data regarding the various application traffic flows in the network. In various embodiments, ABM502may use telemetry data512to track the bitrate of a particular type of application traffic flow (e.g., a VoIP flow associated with Skype, Webex, Jabber, Teams, etc.) at different time granularities (e.g., from a few tens of seconds to a few tens of minutes).

From its bitrate analysis, ABM502may generate a series of bine-indexed bitrate records that quantify the flow bitrate per destination. For instance, an example time-indexed bitrate record may be as follows:

{“timestamp”: “2020-08-03T00:5:00,000Z”,“app_family”: “voice”,“flow_bitrate_per_dest”:{“123.456.789.111”: 2434//number of bits per second and flow of //voice traffic sent to this destination“987.654.321.999”; 1924,“111.222.333.444”; 24043}

ABM502may produce records such as the one above for different application types/families and/or destinations. As shown above, the flow_bitrate_per_dest entry maps the number of bytes per flow marked as voice traffic for a given destination IP. Note that the destination may also be an entire subnet, without loss of generality.

In various embodiments, the time-indexed nitrate records produced by ABM502may be used in either or both of the following ways:They may be stored in a datalake to be processed (in batch) to train machine learning algorithms.They may be used as a stream of input data to an inference engine, such as QoE inference engine506, that performs live estimations of the QoE of the application (e.g., voice) for different destinations.

In some embodiments, ABM502may be configured to filter only relevant applications for which it is possible to determine a proper bitrate mode. In other words, ABM502may operate in conjunction with throughput mode detector (‘I’MD)504, to focus solely on applications that have well-defined modes. In a further embodiment, ARM502may be responsible for detecting situations in which data is missing from telemetry data512or when telemetry data512is unreliable (e.g., if the source router is dropping telemetry records, etc.), to avoid polluting the dataset with corrupted data.

Another potential component of QoE estimation process248may be throughput mode detector (TMD)504, in various embodiments. In general, TMD504may be a machine learning-driven component that uses the data produced by ABM502to detect the relevant modes of throughput for a given application. To achieve this, TMD504may rely on a large amount of historical telemetry data512, to discover the so-called “mode” (e.g., the predominant value in a sample) of the per-flow bitrate observed for a given application. Indeed, some applications are expected to transmit at well-defined bitrates, when healthy. For instance, Skype typically requires between 30 kbps (minimum) and 100 kbps (ideal) for a voice call, and between 1.2 Mbps and 1.5 Mbps for a high-definition (HD) video call.

One observation herein is that the QoE of an application whose traffic is conveyed via a certain network path can be inferred based on the bitrate of the traffic. For instance, given the bitrate of video calls observed along a given path, it can be inferred as to whether the experience on this link is poor (low bitrate) or good (high bitrate). Many other effects can come into play, however, such as the number of participants in the call, the use of content sharing, the configuration of the client, and the like. However, observing the mode of many flows along a given path is likely to give a good indication of the QoE. Indeed, when some segment/link along the path is unable to meet the application requirements, all flows will fall back to a degraded mode of operation and a lower bitrate. Accordingly, in some embodiments, TMD504may identify clusters or modes in the historical distribution of per-flow bitrates and associate them with different QoE measures.

As would be appreciated, voice application and video application traffic are referred to herein for purposes of illustrating the techniques herein, the same principles can be applied to any type of application whose bitrate varies as a function of experience.

FIG. 6illustrates an example plot600of a per-flow bitrate observed in a live network. As shown, the per-flow bitrate distribution in plot600was constructed from a 15-day dataset for Skype application traffic, with plots shown for the average bitrate when the median loss exceeds 1% and for the average bitrate when the median loss is less than or equal to 1%.

From plot600, it can be seen that there are optimal (loss <1%) and degraded (loss >1%) modes for voice-only calls, standard video calls, and HD video calls. Indeed, the application may adjust the bitrate, automatically, when the call quality degrades. Thus, there is a correlation between the bitrate and the QoE of the application, in many cases. Note in plot600that HD video calls almost never happen for high losses, as Skype automatically downgrades the call quality to standard (or disables the video, to altogether), in such situations.

Referring again toFIG. 5, TMD504may automatically find and rank modes of the distribution by using data-driven approaches. In one embodiment, such modes can be simply treated as the local maxima in the estimated distribution. TMD504can estimate this distribution using an appropriate technique, such as kernel density estimation (KDE) techniques, or even by simply computing the traffic histogram. TMD504can then identify the maxima using a trend change detection algorithm or the like. Note, however, that this approach is quite sensitive to the choice of parameters, such the kernel for KDE, or bin size/number of bins for histograms.

In another embodiment, TMD504may estimate the different throughput modes, directly. Here, the idea is to make use of more robust techniques relying on some prior knowledge about the number and shape of the modes. Such techniques include, but are not limited to, k-means. Gaussian Mixture Models (GMMs), some other distribution mixture models, and other similar parametric distribution estimation techniques. What is in common among the approaches belonging to this family of algorithms is the fact they rely on two key assumptions, both easy to validate in practice:The modes of operation are distributed according to some known distribution, e.g. gaussian, heavy tailed, positively/negatively skewed, etc.The number of underlying modes is known.

While the first assumption above is easy to validate via simple data analysis and statistical tests, the second assumption is often grounded on the specifics of the given family of application of interest. Indeed, the second assumption is often a function of the specifics of the application itself and how it operates, such as supporting a voice-only model, one or more video modes, a multicast mode, and the like.

Next, TMD504may match the throughput modes across the different regimes of loss. To do so, TMD504may simply rank the modes and match them based on their position in the ranking, the first with the first, the second with the second, and so on. This matching can be seen inFIG. 6for the different pairs of operation, which can be used to define whether the bitrate corresponds to a ‘good’ QoE or a ‘degraded’ QoE for the application. For instance, there is a first peak for voice-only traffic when the median loss is less than or equal to 1% and a second peak for voice-only traffic when the median loss is greater than 1%. In such a case, TMD504may treat the former as indicative of a good QoE and the latter as indicative of a bad QoE.

by the colors of the arrows. In this way we can define pairs of mode which will be respectively indicate whether the bitrate corresponds to a good or a degraded QoE. The so obtained pairs of modes are passed on to the next component.

Another potential component of QoE estimation process248is QoE inference engine (QIE)506, which may take as input both the stream of bitrate records from ABM502and the list of pairs of throughput modes provided by TMD504. In various embodiments, QIE506may compare the measured traffic against the pair(s) of modes, to infer the QoE of the application for a given destination or set of destinations. For illustrative purposes, let this be expressed as a score L, which denotes the likelihood that the measured traffic is flowing in a degraded bitrate regime. The relationship between flow_bitrate_per_dest from ABM502, the modes identified by TDM504, and the output score L may be defined as an expert-designed heuristic or, alternatively, as a regression algorithm that takes as input the statistical moments of the per-flow bitrate distribution for one or more destinations and the modes produced by TDM504, and produces the score L. In this latter embodiment, the regression could be trained on synthetic data in a lab, based on labels set by experts, or obtained from real users, in various embodiments. Note that the regression model may also be based on a large dataset that spans multiple customers, networks, regions, and/or hardware versions.

As shown, QIE506may provide the resulting QoE estimates516(e.g., L scores) to other components, such as routing process244, a user interface for review by an administrator, or the like. For instance, QIE506may indicate to routing process244the application and destination for which the QoE is degraded. Routing process244may use this information, for instance, to implement a forecasting engine that predicts the evolution of the QoE and initiates predictive routing driven directly by the QoE. In other to words, if the forecasting model of routing process244predicts an upcoming drop in the QoE for an application, it may initiate rerouting of its traffic, before the drop occurs.

In some embodiments, QoE estimation process248may also include QoE validation engine (QVE)508, which is responsible for validating the quality/accuracy of QoE estimates516from QIE506. As shown, QVE508may do so by querying, when available, application programming interfaces (APIs) of the application(s)514whose traffic is being assessed (e.g., Webex, Skype, etc.), to validate that the likelihood L produced by QIE506is indeed correlated with the quality of service observed from direct feedback from the users (e.g., via staffing of calls) or from internal metrics (e.g., codec degradations, etc.). QVE508can use this data to then adjust the heuristics or re-train the regression algorithm of QIE506, to associate bitrate modes to QoE scores.

FIG. 7illustrates an example simplified procedure for estimating the QoE of an application, in accordance with one or more embodiments described herein. For example, a non-generic, specifically configured device (e.g., device200), such as a networking device (e.g., a router, an SDN controller for an SD-WAN, etc.), or a device in communication therewith, may perform procedure700by executing stored instructions (e.g., QoE estimation process248and/or routing process244). The procedure700may start at step705, and continues to step710, where, as described in greater detail above, the device may calculate a distribution of bitrates associated with an application whose traffic is conveyed via one or more paths in a network. For instance, the device may generate time-indexed bitrate records for the application based on telemetry data regarding the traffic (e.g., Netflow data, etc.) and potentially on a per-flow, per-destination basis. In turn, the device may use these bitrate records to calculate the distribution of the bitrates associated with the application. In some instances, each distribution may be associated with a range of values for a network measurement, such as different ranges of loss observed for the application traffic.

At step715, as detailed above, the device may detect throughput modes of the application, based on the one or more distributions of bitrates associated with the application. In some embodiments, the device may do so by estimating the throughput modes from the one or more distributions, based on a predefined number of throughput modes for the application. In further embodiments, the device may do so by identifying local maxima of the distribution(s) as throughput modes.

As step720, the device may associate each throughput mode with a quality of experience label, to form a plurality of pairs of throughput modes and quality of experience labels, as described in greater detail above. For instance, the device may label a particular throughput mode as representing acceptable or unacceptable QoE for the application.

At step725, as detailed above, the device may estimate a quality of experience metric for the application, based on a bitrate of the application and the plurality of pairs of throughput modes and quality of experience labels. In some embodiments, the device may initiate, based on the quality of experience metric estimated for the application, a rerouting of traffic of the application from the one or more paths in the network to one or more other paths (e.g., by providing the estimated QoE to a router, etc.). In further embodiments, the device may use the quality of experience metric estimated for the application to train a forecasting engine configured to forecast the quality of experience metric and proactively reroute traffic of the application, based on that forecast. In yet other embodiments, the device may retrieve a quality of experience measurement from the application and compare the quality of experience measurement from the application to the quality of experience metric estimated for the application. Based on the comparison, the device may initiate retraining of its QoE estimation mechanism, such as when the actual QoE and estimated QoE differ by a threshold amount. Procedure700then ends at step730.

The techniques described herein, therefore, allows for the estimation of the QoE of an application whose traffic is conveyed via a network, by observing the bitrates of the application. Doing so avoids having to rely on actively sending QoS probes along the path(s) of the traffic and comparing the resulting QoS metrics to static SLA templates, which is often an inaccurate way of estimating the QoE of an application. In contrast, the is techniques herein leverage the observation that different throughput modes may exist for the application, allowing inferences about the QoE to be made, directly. Further, the techniques herein can be used to implement proactive routing in the network, such as when the QoE of the application is predicted to degrade over a particular path in the network.

While there have been shown and described illustrative embodiments that provide for estimating the QoE of an application, it is to be understood that various other adaptations and modifications may be made within the spirit and scope of the embodiments herein. For example, while certain embodiments are described herein with respect to using certain models for purposes of estimating QoE metrics, the models are not limited as such and may be used for other types of predictions, in other embodiments. In addition, while certain protocols are shown, other suitable protocols may be used, accordingly.