Bit-rate extraction for encrypted video

A method includes monitoring a media stream that is streamed over a network at a given media bit-rate in a sequence of traffic bursts. Respective data volumes of one or more traffic bursts of the sequence are estimated, and the given media bit-rate is derived from the estimated data volumes.

FIELD OF THE INVENTION

The present invention relates generally to communication analysis, and particularly to methods and systems for bit-rate estimation.

BACKGROUND OF THE INVENTION

Adaptive Bit-Rate (ABR) is a multimedia streaming technique, in which multimedia content is encoded in advance at several predefined bit rates and divided into segments. The content is streamed to a client, while adaptively selecting the bit rate to be streamed depending, for example, on the communication channel conditions en-route to the client. ABR can be used over various communication protocols, such as Hyper-Text Transfer Protocol (HTTP) and HTTP-Secure (HTTPS).

SUMMARY OF THE INVENTION

An embodiment of the present invention that is described herein provides a method including monitoring a media stream that is streamed over a network at a given media bit-rate in a sequence of traffic bursts. Respective data volumes of one or more traffic bursts of the sequence are estimated, and the given media bit-rate is derived from the estimated data volumes.

In some embodiments, estimating the data volumes and deriving the given media bit-rate are performed without decoding content of the media stream. In some embodiments, the media stream is encrypted, and estimating the data volumes and deriving the given media bit-rate are performed without decryption of the media stream.

In some embodiments, estimating the data volumes includes identifying respective start times of the traffic bursts, and distinguishing between the traffic bursts using the identified start times. In an example embodiment, identifying the start times includes interpreting upstream packets as requests that are each followed by a respective traffic burst.

In another embodiment, estimating the data volumes includes distinguishing between a buffering phase and a steady-state phase of the media stream, and assessing the data volumes during the steady-state phase. In yet another embodiment, estimating the data volumes includes distinguishing between a buffering phase and a steady-state phase of the media stream, and assessing the data volumes during the buffering phase.

In some embodiments, the method includes calculating a performance metric of the media stream from the estimated data volumes of the traffic bursts. Calculating the performance metric may include calculating at least one of a start delay, an occurrence of one or more stalls and a length of the stalls in the media stream. In an embodiment, the traffic bursts convey respective content segments having a predefined playback duration, and deriving the given media bit-rate includes, when the media stream is subject to congestion in the network, estimating the given media bit-rate based on the playback duration.

In a disclosed embodiment, the method includes estimating the data volumes and deriving the given media bit-rate only in response to verifying that the media stream matches an Adaptive Bit-Rate (ABR) traffic pattern. In an embodiment, deriving the given media bit-rate includes verifying that the data volumes of the two or more traffic bursts differ from one another by no more than a predefined difference. In another embodiment, deriving the given media bit-rate includes selecting a media bit-rate that best matches the estimated data volumes from among a predefined set of media bit-rates.

There is additionally provided, in accordance with an embodiment of the present invention, an apparatus including an interface and a processor. The interface is configured to receive a media stream that is streamed over a network at a given media bit-rate in a sequence of traffic bursts. The processor is configured to estimate respective data volumes of one or more traffic bursts of the sequence, and to derive the given media bit-rate from the estimated data volumes.

DETAILED DESCRIPTION OF EMBODIMENTS

Overview

In networks that carry video streaming traffic, it is often useful for a third party, other than the server and the client, to estimate the media bit rate of video streams. For example, an operator of a cellular communication network may use the estimated media bit-rates of video streams as an indication of the video quality experienced by clients. This indication can be used, for example, in allocating bandwidth among clients or for applying various Quality-of-Service (QoS) policies.

In this context, it is important to distinguish between media bit-rate and communication bit-rate. The term “media bit-rate” refers to the bit-rate of the actual media (e.g., video) content that is provided to the client. Generally, high media bit-rate corresponds to high media quality, and vice versa. The terms “communication bit-rate” or “traffic bit-rate” refers to the bit-rate of the communication traffic that carries the media. The communication bit-rate is not directly related to the media bit-rate. For example, for a given media bit-rate, the communication bit-rate may vary depending on the conditions of the communication channel to the client.

In some cases it is simple to extract the media bit-rate from the traffic. In many practical cases, however, a third party cannot measure the media bit-rate directly, e.g., because the video content is not accessible. For example, in many cases the video stream is encrypted and cannot be decoded by third parties. A common use case of this sort is Adaptive Bit-Rate (ABR) video transferred over HTTP-Secure (HTTPS).

Embodiments of the present invention that are described herein provide improved methods and systems for estimating the media bit-rate of multimedia streams. In some embodiments, a bit-rate extraction system monitors media streams, e.g., ABR streams, and estimates their media bit-rates without decoding or otherwise accessing the actual video content. In some embodiments the system estimates the media bit-rates by analyzing the characteristic traffic pattern of ABS streaming.

In ABR, the video content is encoded in advance at several media bit-rates, and segmented into segments of constant playback time (playback duration). The video is streamed to a client segment-by-segment, at a media bit-rate that can be chosen to match the client capabilities or channel conditions. The communication traffic carrying an ABR stream has a characteristic initial buffering phase, followed by a steady-state sequence of traffic bursts.

In some embodiments, the system identifies an ABR stream by detecting the above-described traffic pattern, distinguishes between the individual traffic bursts, estimates the data volumes in one or more of the traffic bursts, and derives the media bit-rate of the ABR stream from the estimated data volumes.

Several implementation examples and variations of the disclosed techniques are described below, such as methods for distinguishing between traffic bursts, and for distinguishing between the buffering phase and steady-state phase.

The disclosed techniques extract the media bit-rates of ABR video streams based solely on traffic volume measurements, without any sort of access or decoding of the video content. As such, these techniques are able to measure the media bit-rates even when the video stream, including both content, metadata and signaling, is encrypted.

Although the embodiments described herein refer mainly to ABR streams, the disclosed techniques are applicable to non-ABR streams, as well. Although the embodiments described herein refer mainly to video streaming, the disclosed techniques are applicable to other types of multimedia streams, such as audio streams.

System Description

FIG. 1is a block diagram that schematically illustrates a communication network20that uses video bit-rate extraction, in accordance with an embodiment of the present invention. In network20, a cellular communication network24serves multiple user terminals referred to as User Equipment (UEs)28. The UEs may comprise, for example, cellular phones or mobile computing devices.

Among other communication services, network24transfers ABR video streams to UEs28, to be displayed to users. In the description that follows, the terms “UE,” “terminal,” “client” and “user” are used interchangeably. In the present example, the ABR video streams are streamed from a video server32, across a network36such as the Internet, and over cellular network24to the clients. The video server may comprise, for example, a Web-site such as YouTube or Netflix that provides video streaming.

At least some of the ABR video streams are encrypted, e.g., transferred over HTTPS. In such cases, the entire stream, including video content, metadata and bidirectional signaling, is encrypted. Because of the encryption, a given ABR stream is inaccessible to third parties other than video server32and the intended UE28. A third party may comprise, for example, the service provider or operator of cellular network24.

In some embodiments, a bit-rate extraction system40estimates the media bit-rates of ABR video streams transferred over network24, even though the video streams are inaccessible due to encryption. In the present example, system40is operated by the operator of network24, who uses the estimated media bit-rates for assessing and acting upon the video quality experienced by various users. Alternatively, however, system40can be operated by any other suitable party for any other suitable use.

In the embodiment ofFIG. 1, system40comprises an interface44and a processor48. Interface44is configured to monitor some or all of the video streams transferred over network24. Processor48is configured to carry out the bit-rate estimation methods described herein, and possibly act upon the estimated bit-rates.

The configurations of network20and system40shown inFIG. 1are example configurations, which are chosen purely for the sake of conceptual clarity. In alternative embodiments, any other suitable configuration can be used for network20and/or bit-rate extraction system40. The different elements of system40may be implemented using suitable hardware, e.g., using one or more Application-Specific Integrated Circuits (ASICs) or Field-Programmable Gate Arrays (FPGAs), using software, or using a combination of hardware and software elements.

Bit-Rate Estimation for Encrypted ABR Streams

In a typical ABR implementation, the video content is encoded in advance at several predefined media bit-rates. Each replica of the video content is then segmented into a sequence of segments having a constant playback time (or playback duration), and stored in the video server. A selected replica, having a media bit-rate that matches the client and/or communication channel, is streamed from the video server to the client. The segment edges are typically synchronized among the different replicas, so that it is possible to switch from one replica to another (i.e., from one media bit-rate to another) seamlessly during playback.

In an example embodiment, the video is encoded at three media bit-rates—300 Kbps, 800 Kbps and 1.2 Mbps. Typical segment lengths are between two and ten seconds. Alternatively, however, any other suitable bit-rates and segment lengths can be used.

When an ABR stream is streamed over network24, e.g., over HTTPS, the resulting communication traffic has a characteristic temporal pattern.

FIG. 2is a graph showing the characteristic temporal traffic pattern of ABR video stream, in accordance with an embodiment of the present invention. The graph illustrates the communication bit-rate as a function of time. As explained above, the observable bit-rate is the communication bit-rate, which is not directly related to the media bit-rate.

An ABR session typically begins with an initial phase50, which is referred to as a buffering phase. In the buffering phase, the client typically requests several segments consecutively, in order to initially fill its buffer and decide on the appropriate steady-state media bit-rate. Therefore, the buffering phase is typically characterized by a long initial traffic burst.

After initially filling its buffer and choosing the appropriate media bit-rate, the client typically requests one segment at a time. The client typically requests a new segment when its buffer reaches a certain low threshold. This ongoing process is referred to as a steady-state phase, and is characterized by a sequence of traffic bursts54. Successive traffic bursts54are separated by idle periods. The time between successive traffic bursts (marked58in the figure) is equal to the constant playback time of the video segments.

Changes of media bit-rate during the steady-state phase are usually rare, because the rate switch is noticeable by the user. The client will therefore typically attempt to choose the best-performing media bit-rate during the buffering phase, and keep the media bit-rate constant during the steady-state phase.

Nevertheless, the media bit-rate in a given stream may change during the steady-state phase, for example in response to changes in the communication channel relative to the buffering phase. The techniques described herein are applicable regardless of whether the media bit-rate is constant, changes rarely or changes frequently during the steady-state phase.

In some embodiments, processor48of system40identifies the above-described traffic pattern of the ABR stream, and estimates the media bit-rate of the stream based on the pattern. In an embodiment, processor48measures the respective data volumes contained in one or more traffic bursts54, and derives the media bit-rate from the measured data volumes.

If the number of bits in a traffic burst is X bytes, and the video segment playback time is T seconds, then the media bit-rate is BR=<X·8/T> bits/second (bps). The < > operator denotes averaging over multiple traffic bursts. Thus, processor48may measure the number of bits in a traffic burst (X) over several traffic bursts54, and calculate BR using the known value of T.

In order to perform the calculation above, processor should typically distinguish between consecutive traffic bursts, e.g., by identifying the start times of segments58. In a typical ABR session, the client requests each new segment by sending a GET message to video server32.

It is possible in principle to identify the GET messages and define the time of each GET message as the start time of a new segment. When the ABR stream is encrypted, however, the GET messages are also encrypted and cannot be decoded by system40. Thus, in some embodiments, processor48interprets all upstream messages (messages from the client to the video server) as GET messages, and defines the interval between successive upstream messages as a respective segment.

In these embodiments, processor48measures the data volume between successive upstream messages, and treats this data volume as the data volume of a respective traffic burst54. In alternative embodiments, processor may distinguish between segments, and thus between traffic bursts, in any other suitable way. For example, the processor may identify the idle periods between bursts.

In some embodiments, processor48identifies buffering phase50, as well, using the upstream GET messages. In an example embodiment, the processor may identify several GET messages (or simply several upstream messages) with little or no idle time between them. Such a pattern is highly indicative of the initial buffering phase.

When evaluating the media bit-rate using the BR=<X·8/T> relation, it is important to verify that the traffic bursts being averaged all have the same media bit-rate. Otherwise, e.g., if the media bit-rate has changed during the averaging period, the estimation is likely to be erroneous. In some embodiments, processor48avoids this situation by verifying that the data volumes in the bursts being averaged differ from one another by no more than a predefined difference. If this condition is not met, the processor may choose a different set of bursts for estimating the media bit-rate.

Typically, video server32uses a relatively small set of known predefined media bit-rates from which the client can select, e.g., three or four rates. This information can be used by processor48to improve performance. For example, after measuring the average data volume per traffic burst, processor48may choose the predefined media bit-rate that best matches the average data volume, from among the predefined media bit-rates in the set.

In some embodiments, processor48assesses the media bit-rate by assessing the data volumes during the steady-state phase. For example, it may be known (at least for a particular client type and a particular server) that the client always requests N segments in the buffering phase. Using this information, processor48may measure the data volume during the entire buffering phase and derive the media bit-rate from this measurement.

Typically, processor48estimates the media bit-rate separately per traffic burst. Evaluating the burst-by-burst behavior of the media stream enables processor48to calculate various metrics of interest. One example metric is the start delay—The time that elapses from the time the user sends the initial request to view the stream until the first image frame is displayed on the UE. The start delay can be derived, for example, from the buffering phase, or it can be calculated heuristically. For example, processor48may estimate the length of time needed for downloading ten seconds of content (or any other suitable playback duration), under an assumption that this content size is the size needed to start playing the stream.

Another metric of interest, which can be assessed from the burst-by-burst behavior of the streams, is the occurrence of stalls or freezes. A stall occurs when the UE is unable to download sufficient content to play back. For example, if thirty seconds have elapsed but the UE downloaded content for a playback duration of only 29.5 seconds, then a stall would occur. In this example, the thirty-second duration can be measured directly, and the 29.5-second duration can be inferred by processor48from the traffic bursts and respective bitrates.

Yet another metric of interest is the lengths of the stalls. Once a stall occurs, the streaming typically reverts back to the buffering phase. At this point, the same measurement of start Delay can be used by processor48to measure the length of the stall.

Alternatively, processor48may estimate various other suitable performance metrics of the media stream based on the estimated data volumes (and thus the media bit-rates) of the respective traffic bursts.

It should also be noted that the streaming pattern ofFIG. 2is typical of a non-congested network. In practice, the streaming process encounters congestion in network36and/or24. In some embodiments, processor48is able to estimate the media bit-rate in the presence of congestion, as well. Such estimation is typically based on the knowledge of T, the video segment playback time used by video server32.

In some embodiments processor48may measure T, as explained above, during non-congested periods, and apply the measured T during congestion. Alternatively, processor48may assume typical values of T that are known to be used by a particular service (e.g., YouTube). Since each traffic burst is downloaded between two consecutive GET messages (and often each traffic burst occurs on a different TCP connection), it is possible for processor48to calculate the media bit-rate under congestion from T.

FIG. 3is a flow chart that schematically illustrates a method for bit-rate extraction, in accordance with an embodiment of the present invention. The method begins with interface44of system40receiving an encrypted video stream, at an input step60. (The description ofFIG. 3refers to a single stream for the sake of clarity. In real life, system40typically processes multiple streams in parallel.)

Processor48of system40checks whether the stream is an ABR video stream, at an ABR verification step64. The processor may use the characteristic temporal pattern described above for this purpose. If the stream is not an ABR stream, processor48reverts to an alternative media bit-rate estimation method, at an alternative estimation step68. Such alternative method may apply, for example, to client pacing or server pacing streaming modes.

If step64concludes that the monitored stream is indeed an ABR video stream, processor48proceeds to estimate the media bit-rate of the stream using the disclosed techniques. At a phase identification step72, processor48distinguishes between the buffering phase and the steady-state phase. The processor then identifies the start times of segments58of the steady-state phase, at a segment identification step76. At a data measurement step80, processor48measures the data volume contained in each of one or more of traffic bursts54.

Processor48then estimates the media bit-rate of the ABR stream based on the data volumes of the respective traffic bursts, at a bit-rate derivation step84. At a selection step88, processor48selects one of the predefined media bit-rates supported by server32that is closest to the media bit-rate estimated at step84. In some embodiments, steps84and88can be combined, e.g., by using a direct mapping of data volume per burst to media bit-rate.

Although the embodiments described herein mainly address video streams, the methods and systems described herein can also be used in other applications, such as in bit extraction of audio and other types of media. Moreover, although the embodiments described herein refer mainly to encrypted streams, the disclosed techniques can also be used for measuring the media bit-rates of non-encrypted streams. For example, measuring the media bit-rate based on traffic pattern may be simpler and more computationally efficient than decoding the media content. Furthermore, since the disclosed techniques do not access the media content, they better protect user privacy.