# SDP-Based Quality Adaptation and Performance Prediction in Adaptive Streaming of VBR Videos

Authors: Thoa Nguyen; Thang Vu; Nam Pham Ngoc; Truong Cong Thang
Journal: Advances in Multimedia
 (2017)

---

## Abstract

Recently, various adaptation methods have been proposed to cope with throughput fluctuations in HTTP adaptive streaming (HAS). However, these methods have mostly focused on constant bitrate (CBR) videos. Moreover, most of them are qualitative in the sense that performance metrics could only be obtained after a streaming session. In this paper, we propose a new adaptation method for streaming variable bitrate (VBR) videos using stochastic dynamic programming (SDP). With this approach, the system should have a probabilistic characterization along with the definition of a cost function that is minimized by a control strategy. Our solution is based on a new statistical model where the future streaming performance is directly related to the past bandwidth statistics. We develop mathematical models to predict and develop simulation models to measure the average performance of the adaptation policy. The experimental results show that the prediction models can provide accurate performance prediction which is useful in planning adaptation policy and that our proposed adaptation method outperforms the existing ones in terms of average quality and average quality switch.


---

## Body

## 1. Introduction

Nowadays, video services are increasingly popular on the Internet. According to a recent study and forecast [1], global Internet video traffic will be 80% of the entire consumer Internet traffic in 2019. Besides, HTTP protocol has become a cost-effective solution for video streaming thanks to the abundance of Web platforms and broadband connections [2, 3]. Furthermore, for interoperability of HTTP streaming in the industry, ISO/IEC MPEG has developed “Dynamic Adaptive Streaming over HTTP” (DASH) [4] as the first standard for video streaming over HTTP.DASH requires a video to be available in multiple bitrates and split into small segments each containing a few seconds of playtime. Based on the current network conditions and terminal capacity, the client can adaptively decide a suitable data rate so that stalling is avoided and the available bandwidth is best possibly utilized. If the video is encoded in only one bitrate, either the bitrate is smaller than the available bandwidth resulting in a smooth playback but sparing resources which could be utilized for a better video quality, or the video bitrate is higher than the available bandwidth leading to video stalling. Thus, DASH enables service providers to improve resource utilization and quality of experience (QoE).So far, existing studies have proposed simple heuristics for adapting video at the client. These heuristics can be divided into two types, buffer-based methods and throughput-based methods. The purpose of buffer-based methods is to maintain the stability of the buffer within a certain range to ensure continuous video playback. However, when the bandwidth is drastically reduced, the buffer-based methods may cause sudden change of bitrate [5–8]. Meanwhile, throughput-based methods adaptively decide version based on the estimated throughput. These methods are generally able to react quickly to the throughput variations; the streaming quality, however, may be unstable [9].Recently, several Markov decision-based methods have been proposed to optimize decision making for the streaming client under time-varying network conditions. However, these existing methods mostly focus on constant bitrate (CBR) videos. The authors in [10] are the first to propose an adaptation algorithm in which stochastic dynamic programming (SDP) is employed to find optimal decision policies when streaming VBR videos. The segment requests are ruled by the policies which map a control parameter to every possible state of the system; however, it is limited to videos with weak bitrate fluctuations. To the extent of the authors’ knowledge, in the context of adaptive streaming, there have not been any adaptive streaming methods that could (1) support variable bitrate (VBR) videos with strong bitrate fluctuations and (2) predict the streaming performance with different streaming settings in order to select the optimal one.In this paper, we tackle these challenges by proposing an adaptation method using stochastic dynamic programming. Firstly, we discretize a system including data throughput, buffer level, and bitrate of a VBR video to form the system states. Secondly, we define a cost function that takes into account parameters that affect the subjective perceptual quality of users. In the cost function, the weights are assigned to the difference between data throughput and the bitrate of the next segment, the variance of the buffer from its optimal value, and the quality switch of the video. Finally, we construct an infinite horizon problem (IHP) and solve it to find the optimal policies for all system states. The role of a policy is mapping the control parameter (i.e., the version of the video) to every possible state of the system. This paper is an extended work of our preliminary study in [11]. The extension in this work is multifold. First, we predicted the CDF of the requested versions in a streaming session, so the maximum version could be decided for the streaming session. Second, we predicted CDF of the buffer levels to know the variance of the buffer level under the fluctuations of the network. Finally, we also evaluated the proposed method in the online context, where the statistics of bandwidth is updated periodically. Besides, we compared the performance prediction results with measurement ones in in both offline and online contexts.A policy is optimal when it minimizes an average cost. Based on the obtained policies and the constructed system model, we develop mathematical models that could predict the streaming performance for the new streaming session including average video version, average version switch per segment, average buffer, and average underflow probability. Experiments are conducted to verify the mathematical models by comparing the predicted performance obtained from the models and the measured performance. The proposed method is evaluated in two contexts (1) offline context using statistics of a history bandwidth trace and (2) online context using bandwidth statistics of previous video segments.The paper is organized as follows. Section2 briefly reviews related work. Section 3 describes the system and the modeling of the system in detail. Section 4 presents the formulation of the IHP which is solved by SDP. The performance prediction is given in Section 5. Section 6 presents the experimental results and discussions. Finally, Section 7 concludes our work.


## 2. Related Work

In recent years, many heuristic adaptation methods for adaptive streaming have been developed (e.g., [5–9, 14]). An extensive evaluation of typical adaptation methods has been carried out in [15]. Though these methods prove to be effective in their specific settings, they cannot tell quantitatively the streaming performance with different system settings. Furthermore, most of them focus on CBR videos.For streaming VBR video, several adaptation methods have been proposed based on the sensibility of the receiver’s buffer [6, 16]. Dubin et al. [16] propose an adaptation logic that supports its bandwidth estimation decisions based on the client buffer redundancy. This method considers the fluctuations of mobile network without emphasizing the characteristics of bitrate fluctuation of VBR videos, which leads to the lack of smoothness. In [6], a partial-linear trend prediction model is developed to estimate the trend of client buffer level variation. The client will continue to have the current version for the next segment when the estimated buffer level has no significant change. The drawback of this method is the sudden version switch when the actual buffer level drops dramatically. In [14], the authors propose an adaptive logic for VBR videos based on bitrate estimation. The method demonstrates an effective adaptation behavior as it keeps the buffer at a stable and high level; however, it is still qualitative and has no mechanism to balance the streaming performance.Several recent studies have proposed mathematical model based adaptation methods in streaming video [17–21]. A mathematical model is proposed in [17] to calculate the underflow probability of VBR streaming under VBR channel based on initial delay and maximum buffer. However, this work only considered VBR videos with one version and constant play-out curve without developing any adaptive logic. Meanwhile, Kang et al. [18] present a no-reference, content-based QoE estimation model for video streaming service over wireless networks using neural networks. Nonetheless, neural networks are computationally complex and require large training data and long training time. Besides, Xiang et al. [19] propose a rate adaptation method using the Markov Decision Process to obtain an optimal streaming strategy for VBR videos. Nevertheless, their proposal is not able to adapt to the real bandwidth changes and has no performance prediction. The prediction of streaming performance was first proposed for streaming CBR videos in [20] and VBR videos in [21] by Liu et al. In their studies, a video session is divided into subsessions. With a given target rebuffering probability, the video bitrate/the average video bitrate for each streaming subsession is predicted when streaming CBR videos/VBR videos, respectively. The results show that the average actual rebuffering probability achieved by these methods is reasonably close to the target. However, they have not done any assessments in terms of video quality and video quality switch, so the QoE can be affected.Recently, SDP has been known as an effective technique for solving optimization problems in video streaming [10, 22, 23]. For instance, the authors in [22] apply SDP to find the optimal policy for choosing sending rates when streaming on-demand scalable VBR videos over wireless network. Nevertheless, they have not considered the effect of channel-state aware adaptation. Meanwhile, García et al. [10] construct an infinite horizon problem and apply SDP to solve it specifically for HTTP adaptive streaming. They propose a channel model in which transitions are only possible to adjacent states, with equal probabilities, which is only suitable for the stable bandwidth, with little fluctuation. In addition, by observing the histograms of segment size encoded with VBR, the authors assume that segment size (which is proportional to segment bitrate) can be modeled through a discrete Gaussian distribution. Then, the probability distribution of segment sizes is used to calculate transition probabilities. However, the probability distribution of segment sizes is not taken into account in the cost function. Instead, the average bitrate representing the bitrate for each version is used. This is only reasonable when the deviations of segment sizes (i.e., segment bitrates) are small. In another study, Xing et al. [23] use SDP to find the optimization for Advance Video Coding content when streaming through several wireless connections at the same time. They offer a cost function in terms of QoE, but the computational complexity of their model is significantly caused by eight system variables in each state.The SDP-based method proposed for HTTP streaming in this paper isdifferent from the previous studies in several points. First, our method considers an actual time-varying bandwidth of mobile networks. Second, the drastic bitrate fluctuations of actual VBR videos are effectively supported. Third, we develop mathematic models to predict the performance of a streaming subsession, which helps to select the maximum allowed version parameter in advance.


## 3. System Modeling

### 3.1. System Overview

Figure1 shows the functional diagram of a general DASH system consisting of a server and a client. The server holds the media files with different quality versions. Each version is further divided into small segments. The client has the information about the characteristics and locations of media segments and can request any of them during a streaming session. For the next segment, the client makes a decision of what video version to request based on current status of client buffer and data throughput to provide the best streaming experience possible. In this paper, the segment selection policy for the client aims at maintaining the client buffer at a reasonable level while balancing between average version (i.e., average quality) and average version switch (i.e., quality variations).Figure 1
General DASH system.In order to apply SDP, our system is modeled as a discrete-time stochastic one. Specifically, the timeline is divided into time stages. At each stagek, the system is represented by a state variable sk. When the next segment is completely downloaded at stage k+1, the system moves to the state sk+1. As the system transits to the next state, a certain cost occurs. Besides, the channel, the buffer, and the media are discretized as explained in the following subsection.


### 3.2. Channel, Buffer, and Media Model

In our work, we discretize the bandwidth range intoW levels. The bandwidth trace before and after discretization is shown in Figure 2 withW = 10. With this level of quantization, the quantized bandwidth covers the original bandwidth well. We then create W different bandwidth states BWi(1≤i≤W) from these W bandwidth levels. The value of each bandwidth state is the average of the maximum value and minimum value of the corresponding bandwidth level. To represent the transition from one bandwidth state to another on a bandwidth state space, we use the Markov chain model which has been used widely in previous studies [10, 24, 25].Figure 2
A bandwidth trace obtained from a mobile network [12].Figure3 presents a general Markov-chain model which consists of three bandwidth states. Each state is represented by one data throughput value. There is a transition probability when the bandwidth moves from one state to another after each time step. Thus, by simply extracting the statistics from the bandwidth history, the transition probability between all bandwidth states is generated.Figure 3
General 3-state Markov-chain bandwidth model.Similar to the bandwidth trace, we divide the buffer intoB levels from 0 to Bs, with Bs being the buffer size. In addition, we denote the video version by V and represent the version with lowest quality asV = 1 and the highest quality asV =Vmax assuming that the video hasVmax different versions. The bitrates of different versions of a VBR video are shown in Figure 4.Figure 4
Bitrates of different versions of Tokyo Olympic video [13].Because the segment bitrate of a VBR video version fluctuates very strongly, we divide the bitrate of each version intoI intervals (from interval 1 to intervalI), each of which is represented by its average bitrate value. For example, if a version that has the highest bitrate of 5000 kbps is divided into 10 bitrate intervals, the interval 1 will range from 0 to 500 kbps and its representative bitrate will be 250 kbps.We assume that all versions at a bitrate interval represent a separate video flow. If the current segment bitrate belongs to one interval, the next segment bitrate will also belong to that interval regardless of segment version. With this assumption, we generateI different policy sets for I bitrate intervals. When a segment is completely downloaded, the client measures its bitrate to find the bitrate interval it belongs to. Then, the client will determine the policy set corresponding to that bitrate interval.


## 3.1. System Overview

Figure1 shows the functional diagram of a general DASH system consisting of a server and a client. The server holds the media files with different quality versions. Each version is further divided into small segments. The client has the information about the characteristics and locations of media segments and can request any of them during a streaming session. For the next segment, the client makes a decision of what video version to request based on current status of client buffer and data throughput to provide the best streaming experience possible. In this paper, the segment selection policy for the client aims at maintaining the client buffer at a reasonable level while balancing between average version (i.e., average quality) and average version switch (i.e., quality variations).Figure 1
General DASH system.In order to apply SDP, our system is modeled as a discrete-time stochastic one. Specifically, the timeline is divided into time stages. At each stagek, the system is represented by a state variable sk. When the next segment is completely downloaded at stage k+1, the system moves to the state sk+1. As the system transits to the next state, a certain cost occurs. Besides, the channel, the buffer, and the media are discretized as explained in the following subsection.


## 3.2. Channel, Buffer, and Media Model

In our work, we discretize the bandwidth range intoW levels. The bandwidth trace before and after discretization is shown in Figure 2 withW = 10. With this level of quantization, the quantized bandwidth covers the original bandwidth well. We then create W different bandwidth states BWi(1≤i≤W) from these W bandwidth levels. The value of each bandwidth state is the average of the maximum value and minimum value of the corresponding bandwidth level. To represent the transition from one bandwidth state to another on a bandwidth state space, we use the Markov chain model which has been used widely in previous studies [10, 24, 25].Figure 2
A bandwidth trace obtained from a mobile network [12].Figure3 presents a general Markov-chain model which consists of three bandwidth states. Each state is represented by one data throughput value. There is a transition probability when the bandwidth moves from one state to another after each time step. Thus, by simply extracting the statistics from the bandwidth history, the transition probability between all bandwidth states is generated.Figure 3
General 3-state Markov-chain bandwidth model.Similar to the bandwidth trace, we divide the buffer intoB levels from 0 to Bs, with Bs being the buffer size. In addition, we denote the video version by V and represent the version with lowest quality asV = 1 and the highest quality asV =Vmax assuming that the video hasVmax different versions. The bitrates of different versions of a VBR video are shown in Figure 4.Figure 4
Bitrates of different versions of Tokyo Olympic video [13].Because the segment bitrate of a VBR video version fluctuates very strongly, we divide the bitrate of each version intoI intervals (from interval 1 to intervalI), each of which is represented by its average bitrate value. For example, if a version that has the highest bitrate of 5000 kbps is divided into 10 bitrate intervals, the interval 1 will range from 0 to 500 kbps and its representative bitrate will be 250 kbps.We assume that all versions at a bitrate interval represent a separate video flow. If the current segment bitrate belongs to one interval, the next segment bitrate will also belong to that interval regardless of segment version. With this assumption, we generateI different policy sets for I bitrate intervals. When a segment is completely downloaded, the client measures its bitrate to find the bitrate interval it belongs to. Then, the client will determine the policy set corresponding to that bitrate interval.


## 4. Problem Formulation and Solution

### 4.1. System State

With the system being discretized above, we observe the system state variableskbk,bwk,vk when a video segment is completely downloaded at stage k. Here, bk is the buffer level representing the number of segments available in the buffer, bwk is the bandwidth whose value belongs to {BWi}, and vk is the version index of the downloaded segment. The case where bk=1 corresponds to the buffer underflow event. In each state sk, the system may choose any action a. For our system, an action is basically a decision about the version for the next segment. As there are Vmax versions to choose from, we have totally Vmax possible actions. The system then randomly moves into a new state sk+1 at the next time step,resulting in a corresponding cost Csk,sk+1,a. With each bitrate interval i1≤i≤I, we have a system state set si and a policy set μi. Let N be the number of states in si, and we have(1)N=B∗W∗Vmax.


### 4.2. Transition Probabilities

Since the system is stochastic, which means the system outcome of each actiona is not deterministic, the state transition probability between every two states that depends on action a must be constructed. We denote the probability that state sk will lead to state sk+1 given action a as follows:(2)Psk,sk+1,a=Pr⁡sk+1∣sk,a.Due to the independence among (bk,bwk,vk), we have(3)Pr⁡sk+1∣sk,a=Pr⁡bk+1∣bk,bwk,vk,aPr⁡bwk+1∣bwkPr⁡vk+1∣a.In the right hand side of (3), the first term can be calculated as follows:(4)Pr⁡bk+1∣bk,bwk,vk,a=1,bk+1=ba0,otherwise,where ba is the next buffer level estimated based on the current system status and action a. We calculate ba as follows:(5)ba=bk+1-rabwk+1,where ra is the bitrate of the target version.When the throughput significantly drops, meaning a very low value ofbwk+1, ba could be lower than zero. However, at the beginning of a new stage, there is one segment being downloaded resulting in at least one segment being always in the buffer. Therefore, (5) can be modified as follows:(6)ba=max⁡bk+1-rabwk+1,1.The second term is easily obtained from the bandwidth model. And the third term can be simply calculated by(7)Pr⁡vk+1∣a=1,vk+1=a0,otherwise.Thus, expression (3) can be simplified as follows:(8)Pr⁡sk+1∣sk,a=Prbwk+1∣bwk,bk+1=ba,vk+1=a0,otherwise.


### 4.3. Cost Function

In this section, a cost function is defined to punish the situations that may cause a decrease in users’ QoE. We focus on three objective parameters that affect strongly the subjective perception of the users which are quality level, video stalling, and quality switch. First, the cost function should favor the selected bitrate to be close to the current bandwidth, so it punishes the differenceΔr between the current bandwidth and the bitrate of the next segment selected by action a, with(9)Δr=bwk-ra.Second, to prevent video stalling, the buffer level should never be underflow. We define an optimal buffer level bopt that is a desired value the client should try to keep during a streaming session. When the buffer level is close to bopt, the buffer underflow is avoided. Therefore, the cost function penalizes the deviation Δb of the current buffer level from the optimal buffer level, where (10)Δb=bk-bopt.Third, in order to reduce the quality switches, the cost function should contain the difference Δq between the selected quality and the last one. To punish a QoE reduction because of high quality variations, we define Δq as follows:(11)Δq=a-vk2.Letα,β,γ be the trade-off parameters of three objects, namely, quality level, video stalling, and quality switch, respectively. The cost incurred when the system changes from state sk to state sk+1 given action a can be calculated by (12)Csk,sk+1,a=αΔr+βΔb+γΔq.


### 4.4. Optimization Solution

As the system is discrete and the number of states is large, we can formulate an infinite horizon problem. For every statesk, the most appropriate action a, called policy for state sk, has to be decided so that the mean cost per state is minimum. As mentioned above, our system has I system state sets corresponding to I bitrate intervals, so we have to find I corresponding policy sets. For simplicity, we only present the optimization solution for a general bitrate interval with the system state set s and a corresponding policy set μ. Finding optimal solutions for all bitrate intervals will be done similarly. Mathematically, we have to minimize CA which is the average cost per state obtained after downloading L video segments. CA is calculated as follows:(13)CA=limL→∞⁡1L∑l=1LPsk,sk+1,aClsk,sk+1,a,with Cl(sk,sk+1,a) being the cost incurred after downloading the lth segment. Here, L is the number of state transitions and is also the number of video segments in the session. Based on the standard policy iteration algorithm (PIA) [26], we solve the IHP problem by using an algorithm as presented in Algorithm 1.Algorithm 1:Finding the policy set for one interval.
Input: number of states N,
probability function P,
cost function C
Output:optimal policy for each state μsk
j
=
0 // j is the iteration index
μ
s
k
j
=
0
,
∀
s
k
∈
s
//
μ
s
k
j is policy of state sk at jth iteration
repeat
j
=
j
+
1
Policy evaluation: compute λj,hj(sk) using the below
N
+
1 equations:
h
j
s
N
=
0
λ
j
+
h
j
s
k
=
C
s
k
,
s
k
+
1
,
μ
s
k
j
+
∑
r
=
1
N
P
s
k
,
s
r
,
μ
s
r
j
h
j
s
r,
∀
s
k
∈
s
Policy improvement:find for all sk
μ
s
k
j
+
1
=
arg
⁡
min
a
⁡
C
s
k
,
s
k
+
1
,
a
+
∑
r
=
1
N
P
s
k
,
s
r
,
a
h
j
s
r,
until  λj+1=λj and hj+1sk=hjsk,∀sk∈sApplying PIA of SDP forI bitrate intervals, we would generate I policy sets which serve like a look-up table mapping each state to an optimal action. Thus, the client is able to decide an appropriate version for the next segment based on the current system condition.LetCprob,Ccost be the computational complexity of the calculation of transition probability and the cost from one state to the remaining states, respectively. Let CPIA be the computational complexity of Algorithm 1. The complexity of our model is C=Cprob+Ccost+CPIA. For each interval, each action, and each state, we consider the cost and the transition probability to N-1 remaining states. So the complexity of the calculation of transition probability and the cost is described as follows:(14)Cprob=Ccost=OIVmaxN2.Based on [27], we have(15)CPIA=OIVmaxN.Therefore, the complexity of our model is(16)C=O2IVmaxN2+IVmaxN.


## 4.1. System State

With the system being discretized above, we observe the system state variableskbk,bwk,vk when a video segment is completely downloaded at stage k. Here, bk is the buffer level representing the number of segments available in the buffer, bwk is the bandwidth whose value belongs to {BWi}, and vk is the version index of the downloaded segment. The case where bk=1 corresponds to the buffer underflow event. In each state sk, the system may choose any action a. For our system, an action is basically a decision about the version for the next segment. As there are Vmax versions to choose from, we have totally Vmax possible actions. The system then randomly moves into a new state sk+1 at the next time step,resulting in a corresponding cost Csk,sk+1,a. With each bitrate interval i1≤i≤I, we have a system state set si and a policy set μi. Let N be the number of states in si, and we have(1)N=B∗W∗Vmax.


## 4.2. Transition Probabilities

Since the system is stochastic, which means the system outcome of each actiona is not deterministic, the state transition probability between every two states that depends on action a must be constructed. We denote the probability that state sk will lead to state sk+1 given action a as follows:(2)Psk,sk+1,a=Pr⁡sk+1∣sk,a.Due to the independence among (bk,bwk,vk), we have(3)Pr⁡sk+1∣sk,a=Pr⁡bk+1∣bk,bwk,vk,aPr⁡bwk+1∣bwkPr⁡vk+1∣a.In the right hand side of (3), the first term can be calculated as follows:(4)Pr⁡bk+1∣bk,bwk,vk,a=1,bk+1=ba0,otherwise,where ba is the next buffer level estimated based on the current system status and action a. We calculate ba as follows:(5)ba=bk+1-rabwk+1,where ra is the bitrate of the target version.When the throughput significantly drops, meaning a very low value ofbwk+1, ba could be lower than zero. However, at the beginning of a new stage, there is one segment being downloaded resulting in at least one segment being always in the buffer. Therefore, (5) can be modified as follows:(6)ba=max⁡bk+1-rabwk+1,1.The second term is easily obtained from the bandwidth model. And the third term can be simply calculated by(7)Pr⁡vk+1∣a=1,vk+1=a0,otherwise.Thus, expression (3) can be simplified as follows:(8)Pr⁡sk+1∣sk,a=Prbwk+1∣bwk,bk+1=ba,vk+1=a0,otherwise.


## 4.3. Cost Function

In this section, a cost function is defined to punish the situations that may cause a decrease in users’ QoE. We focus on three objective parameters that affect strongly the subjective perception of the users which are quality level, video stalling, and quality switch. First, the cost function should favor the selected bitrate to be close to the current bandwidth, so it punishes the differenceΔr between the current bandwidth and the bitrate of the next segment selected by action a, with(9)Δr=bwk-ra.Second, to prevent video stalling, the buffer level should never be underflow. We define an optimal buffer level bopt that is a desired value the client should try to keep during a streaming session. When the buffer level is close to bopt, the buffer underflow is avoided. Therefore, the cost function penalizes the deviation Δb of the current buffer level from the optimal buffer level, where (10)Δb=bk-bopt.Third, in order to reduce the quality switches, the cost function should contain the difference Δq between the selected quality and the last one. To punish a QoE reduction because of high quality variations, we define Δq as follows:(11)Δq=a-vk2.Letα,β,γ be the trade-off parameters of three objects, namely, quality level, video stalling, and quality switch, respectively. The cost incurred when the system changes from state sk to state sk+1 given action a can be calculated by (12)Csk,sk+1,a=αΔr+βΔb+γΔq.


## 4.4. Optimization Solution

As the system is discrete and the number of states is large, we can formulate an infinite horizon problem. For every statesk, the most appropriate action a, called policy for state sk, has to be decided so that the mean cost per state is minimum. As mentioned above, our system has I system state sets corresponding to I bitrate intervals, so we have to find I corresponding policy sets. For simplicity, we only present the optimization solution for a general bitrate interval with the system state set s and a corresponding policy set μ. Finding optimal solutions for all bitrate intervals will be done similarly. Mathematically, we have to minimize CA which is the average cost per state obtained after downloading L video segments. CA is calculated as follows:(13)CA=limL→∞⁡1L∑l=1LPsk,sk+1,aClsk,sk+1,a,with Cl(sk,sk+1,a) being the cost incurred after downloading the lth segment. Here, L is the number of state transitions and is also the number of video segments in the session. Based on the standard policy iteration algorithm (PIA) [26], we solve the IHP problem by using an algorithm as presented in Algorithm 1.Algorithm 1:Finding the policy set for one interval.
Input: number of states N,
probability function P,
cost function C
Output:optimal policy for each state μsk
j
=
0 // j is the iteration index
μ
s
k
j
=
0
,
∀
s
k
∈
s
//
μ
s
k
j is policy of state sk at jth iteration
repeat
j
=
j
+
1
Policy evaluation: compute λj,hj(sk) using the below
N
+
1 equations:
h
j
s
N
=
0
λ
j
+
h
j
s
k
=
C
s
k
,
s
k
+
1
,
μ
s
k
j
+
∑
r
=
1
N
P
s
k
,
s
r
,
μ
s
r
j
h
j
s
r,
∀
s
k
∈
s
Policy improvement:find for all sk
μ
s
k
j
+
1
=
arg
⁡
min
a
⁡
C
s
k
,
s
k
+
1
,
a
+
∑
r
=
1
N
P
s
k
,
s
r
,
a
h
j
s
r,
until  λj+1=λj and hj+1sk=hjsk,∀sk∈sApplying PIA of SDP forI bitrate intervals, we would generate I policy sets which serve like a look-up table mapping each state to an optimal action. Thus, the client is able to decide an appropriate version for the next segment based on the current system condition.LetCprob,Ccost be the computational complexity of the calculation of transition probability and the cost from one state to the remaining states, respectively. Let CPIA be the computational complexity of Algorithm 1. The complexity of our model is C=Cprob+Ccost+CPIA. For each interval, each action, and each state, we consider the cost and the transition probability to N-1 remaining states. So the complexity of the calculation of transition probability and the cost is described as follows:(14)Cprob=Ccost=OIVmaxN2.Based on [27], we have(15)CPIA=OIVmaxN.Therefore, the complexity of our model is(16)C=O2IVmaxN2+IVmaxN.


## 5. Performance Prediction

After Section4, we achieve I policy sets corresponding to I intervals of a video bitrate. In this section, we use the Markov chain model to predict the streaming performance for a session. Similar to Section 4.4, this section only presents the performance prediction for a general bitrate interval with a system state set s and a corresponding policy set μ. Predicting performance for all bitrate intervals will be done similarly. After carrying out the calculation for all I bitrate intervals, we take the average values as the final results.The key is to determine the average state probabilitypa=[pa1,pa2,…,paN] with pan(1≤n≤N) being the average probability that the system is at the nth state throughout the streaming session. The probability pa is the average value of state probabilities after downloading l segments pl=pl1,pl2,…,plN,(1≤l≤L). Here, pln(1≤n≤N) is the probability that the system is at the nth state after downloading l segments. From the Markov chain theory, the state probability after downloading l+1 segments can be computed as the product of the state probability after downloading l segments and a transition matrix pl+1=plPTS. Assuming that the initial probability p0 is known. The transition matrix PTS is a N×N matrix which represents the transition probability from state sk to state sk+1. PTS is defined as follows:(17)PTS=Psk,sk+1=Pr⁡sk+1∣sk,a∈μ.The average state probability pa is calculated as follows.(18)pa=∑l=1LplPTSL.Currently, most of the adaptation algorithms developed for HTTP streaming are qualitative in the sense that the performance metrics could only be obtained after the streaming session. In this study, the predicted streaming performance could be calculated based on the average state probability and the information inside every state. Specifically, we mainly focus on the following aspects: bitrate prediction, quality switch prediction, and buffer prediction.


### 5.1. Quality Prediction

The video quality that the users perceive is presented through the selected version. The higher the version is selected, the better the video quality is perceived by the users. Furthermore, setting the maximum version for the streaming session also affects the perceptual quality of the users. Obviously, a very low value of the maximum version may result in a poor perceptual quality while a very high value may increase the chance of buffer underflow. In this section, we predict the quality performance of the streaming session based on the average versionAv that is calculated using (19) and the cumulative distribution function (CDF) of the versions fv(v∈1;Vmax) that is shown in (20). Based on the predicted probability of the versions throughout a streaming session, the maximum version could be decided for the session(19)Av=∑n=1NvnpanN,(20)fv=∑n=1NvnpanN∣v=vn.


### 5.2. Quality Switch Prediction

Quality switch is an important factor affecting the perception of the users. The users often expect a smooth playback with the minimum number of quality switches and small switch amplitude from one segment to the next. We can predict the average version switch per segmentAsw as follows:(21)Asw=∑n=1Nan-vnpanN,an∈μ.


### 5.3. Buffer Prediction

Video stalling is one of the important objective parameters that affect the subjective perception of the users. Stalling occurs when the play-out buffer underruns. To prevent this event, the buffer must be maintained within a safe range. In this session, we evaluate the buffer performance through the average buffer levelAb, the CDF of the buffer level f^(b)(b∈1;Bmax), and the buffer underflow probability Prund (i.e., when the system stays at buffer level 1) which are described as follows:(22)Ab=∑n=1NbnpanN,f^b=∑n=1NbnpanN∣bn<b,Prund=∑n=1NpanN∣bn=1.Ab represents the safety of the buffer. If Ab is small, the buffer level is often in low levels, which may cause playback interruption when the current bandwidth drops dramatically. f^b reflects the variance of the buffer level under the fluctuations of the network. Prund shows the probability that the playback would be interrupted in the streaming session.


## 5.1. Quality Prediction

The video quality that the users perceive is presented through the selected version. The higher the version is selected, the better the video quality is perceived by the users. Furthermore, setting the maximum version for the streaming session also affects the perceptual quality of the users. Obviously, a very low value of the maximum version may result in a poor perceptual quality while a very high value may increase the chance of buffer underflow. In this section, we predict the quality performance of the streaming session based on the average versionAv that is calculated using (19) and the cumulative distribution function (CDF) of the versions fv(v∈1;Vmax) that is shown in (20). Based on the predicted probability of the versions throughout a streaming session, the maximum version could be decided for the session(19)Av=∑n=1NvnpanN,(20)fv=∑n=1NvnpanN∣v=vn.


## 5.2. Quality Switch Prediction

Quality switch is an important factor affecting the perception of the users. The users often expect a smooth playback with the minimum number of quality switches and small switch amplitude from one segment to the next. We can predict the average version switch per segmentAsw as follows:(21)Asw=∑n=1Nan-vnpanN,an∈μ.


## 5.3. Buffer Prediction

Video stalling is one of the important objective parameters that affect the subjective perception of the users. Stalling occurs when the play-out buffer underruns. To prevent this event, the buffer must be maintained within a safe range. In this session, we evaluate the buffer performance through the average buffer levelAb, the CDF of the buffer level f^(b)(b∈1;Bmax), and the buffer underflow probability Prund (i.e., when the system stays at buffer level 1) which are described as follows:(22)Ab=∑n=1NbnpanN,f^b=∑n=1NbnpanN∣bn<b,Prund=∑n=1NpanN∣bn=1.Ab represents the safety of the buffer. If Ab is small, the buffer level is often in low levels, which may cause playback interruption when the current bandwidth drops dramatically. f^b reflects the variance of the buffer level under the fluctuations of the network. Prund shows the probability that the playback would be interrupted in the streaming session.


## 6. Experimental Results

In order to evaluate the proposed system model and performance prediction accuracy, in this section, we perform a number of experiments in both offline and online contexts and compare the performance predicted results with the measurement ones. We also compare our proposed method with two existing ones, namely, the SDP method presented [10] which could obtain the best performance among the SDP methods and the bitrate estimation based method presented [14], which is the best among the qualitative methods.


### 6.1. Experiment Setup

For the simulation, our test-bed consists of a client running Java 8.0 which implements the adaptation and a server running Apache2 which holds the media segments. The client runs on a Window 7 computer with an Intel i5-1.7 GHz CPU and 4 GB memory and the server runs on Ubuntu 12.04LTS (with default TCP CUBIC) with 1 G RAM. The channel bandwidth is simulated using DummyNet [28]. We use the Tokyo Olympic video from [6]. For the video, Vmax=9, and, for the bandwidth, W=10. Since we measure the buffer size and compute the buffer cost in the segment duration unit, we implement our adaptation method with one setting of the segment duration. In our experiments, we select a segment duration of 2 seconds, which is similar to those of [7, 8, 10]. The impacts of segment durations on adaptation performance have been considered in some recent studies [29, 30]. Further evaluation of different segment durations with a fixed buffer size (in seconds) will be reserved for our future work. Maximum buffer level is set to 5 segments (i.e., 10 seconds) and optimal buffer level is set to 4 segments (i.e., 8 seconds).In our method, a streaming provider can adjust the balance between the requirements for high quality level, preventing video stalling, and reducing quality switches by changing the trade-off parametersα,γ, and β. Since selecting an optimal combination of trade-off parameters of the cost function involves solving a hard optimization problem, it will be investigated in our future work. In this paper, we select qualitatively the trade-off parameters of cost function as follows. Initially, we fix β and select the other two parameters. Since we want to prioritize requirement of smooth quality switch, γ is selected so that the contribution of the quality switch cost Δq is higher than that of buffer cost Δb. With parameter α, because the bitrate cost Δr can be up to thousands, parameter α should be small to reduce the contribution of the bitrate cost to the overall cost C. Based on our experience, good empirical values on parameters α,β, and γ are 0.003, 4, and 20, respectively.


### 6.2. Experimental Results

In the first part of the experiment, we evaluate the accuracy of performance prediction in offline context using a given bandwidth trace obtained from a mobile network [12]. In this context, the number of video segments L is 300.Figures5 and 6 show the predicted performance using the formulas presented in Section 5 and the measured performance obtained from the experiments when the maximum allowed version is set to 7, 8, and 9. These figures point out that the prediction results are close to the measurement ones. We can see from Figure 5 that when the maximum version increases the average version as well as the average version switch also increases. Figure 6 shows that, in both prediction and measurement cases, when the maximum version is 7, the underflow probability is almost zero and increases very slowly when the maximum allowed version increases. This analysis implies that setting the maximum version to 8 is reasonable in terms of balancing between average video quality and quality switch; meanwhile setting the maximum version to 7 ensures a very stable streaming experience.Figure 5
Predicted performance and measured performance in terms of average version and average version switch per segment in offline context using a given bandwidth trace.Figure 6
Predicted performance and measured performance in terms of average buffer and underflow probability in offline context using a given bandwidth trace.Figures7, 8, and 9 show the bitrate and version switch behavior of the proposed method in the three cases of maximum version. It can be seen very clearly from these figures that when the maximum version is reduced, the number of version switches (or quality changes) decreases.Figure 7
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 9.Figure 8
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 8.Figure 9
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 7.Table1 shows more detailed statistics of the experimental results in three cases of maximum version. It can be drawn from the table that there is no significant difference between the predicted performance and the measured performance.Table 1
Compare predicted performance and measured performance in offline context using a given bandwidth trace.
Statistics
V
m
a
x
=
9
V
m
a
x
=
8
V
m
a
x
=
7
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.48
8.52
9.59
8.77
9.8
9.68
A
q
7.64
7.88
7.45
7.73
6.48
6.87
A
s
w
0.30
0.18
0.14
0.09
0.03
0.02
P
r
u
n
d
f
l
o
w
0.01
0.00
0.006
0.00
0.00
0.00In the second part of the experiment, we use two history bandwidth traces recorded from two previous streaming sessions of the client. The CDFs of both bandwidths are shown in Figure10.Figure 10
The cumulative distribution function of bw1 and bw2.Bandwidthbw1 is used for calculating the statistical models and bw2 is used in the simulation for measuring performance parameters. Figures 11, 12, and 13 show the bitrate and version switch behavior of the proposed method in the three cases of maximum version. The detailed results are listed in Table 2. Based on these figures and table, we affirm once again that the mathematical performance prediction model agrees well with the measurement.Table 2
Compare predicted performance and measured performance in offline context using a history bandwidth trace for statistics.
Statistics
V
m
a
x
=
9
V
m
a
x
=
8
V
m
a
x
=
7
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.52
8.44
9.61
8.85
9.69
9.65
A
q
7.48
7.92
7.27
7.71
6.80
6.98
A
s
w
0.31
0.19
0.18
0.11
0.08
0.05
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Figure 11
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 9.Figure 12
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 8.Figure 13
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 7.In the third part of the experiment, we consider the online context in which the prediction of the future bandwidth is based on the statistical parameters of all previous segments. Specifically, we divide an entire session into chunks, each of which hasL video segments. We treat each chunk individually as a mini streaming session. Assuming that, initially we have enough statistical data to predict the performance of the first chunk. The prediction of the subsequent chunks will be done based on the previous chunks. At the beginning of each chunk, the bandwidth statistics are updated leading to a recomputation of the policy set and the performance. In the experiment, we set L to 100 video segments. It can be observed from our experiments that it took only several seconds to (re)compute the whole model. Therefore, the computational overhead is about 3%, which is appropriate for the online context. Figures 14, 15, and 16 show the adaptation, bitrate, and version switch behavior of the proposed method in the three cases of maximum version. The predicted performance and the measured performance in the online context in the three cases are presented in detail in Tables 3, 4, and 5. It is very clear that, in the online context, the predicted performance is also very close to the measured performance.Table 3
Compare predicted performance and measured performance in online context withVmax=9.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.31
8.96
8.92
8.45
9.36
8.86
A
q
7.69
8.04
7.89
7.68
7.83
7.72
A
s
w
0.05
0.09
0.04
0.07
0.02
0.05
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Table 4
Compare predicted performance and measured performance in online context withVmax=8.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.53
9.01
9.42
8.63
9.42
8.96
A
q
7.56
7.86
7.65
7.63
7.65
7.67
A
s
w
0.00
0.03
0.01
0.03
0.02
0.04
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Table 5
Compare predicted performance and measured performance in online context withVmax=7.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.77
9.93
9.77
9.52
9.83
9.59
A
q
6.92
7.00
6.92
6.95
6.92
7.00
A
s
w
0.00
0.00
0.00
0.02
0.00
0.00
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Figure 14
Experimental results of the proposed method in online context withVmax = 9.Figure 15
Experimental results of the proposed method in online context withVmax = 8.Figure 16
Experimental results of the proposed method in online context withVmax = 7.Next, we compare our proposed method with the SDP method [10] and the bitrate estimation based method [14] using the same simulation settings as in the first part of our experiment. The experimental results obtained by simulating the SDP method [10] and bitrate estimation based method [14] are shown in Figures 17 and 18, respectively. We can see that both methods provide very fluctuating version switches. The detailed statistics of these adaptation methods with the maximum version set to 9 and our proposed method with the maximum version set to 7, 8, and 9 are shown in Table 6. It can be seen that the performance of the bitrate estimation method is less effective than our method in terms of average version and average version switch. Regarding the SDP method, it is evident that the performance of this method is the worst among all (with low average version and highest average switch per segment). This can be expected because this method was not originally designed for real VBR videos.Table 6
Statistics of different adaptation methods in offline context.
Statistics
SDP methodVmax=9
Bitrate estimation methodVmax=9
Proposed methodVmax=9
Proposed methodVmax=8
Proposed methodVmax=7
A
b
(
s
)
8.62
8.85
8.52
8.77
9.68
A
q
7.40
7.85
7.88
7.73
6.87
A
s
w
0.76
0.43
0.18
0.09
0.02
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00Figure 17
Experimental results of SDP method.Figure 18
Experimental results of bitrate estimation based method.The buffer level curves of the three methods in offline context are shown in Figure19. We can see that all buffer curves imply streaming sessions without any freezes for users. Among these, the SDP method and the proposed method with Vmax=9 provide the most unstable buffers, while the bitrate estimation based method and the proposed method with Vmax=7 provide the most stable buffers. It can be observed from Figures 17 and 18 and Table 6 that the SDP method and bitrate estimation based method result in very fluctuating version curves, and the proposed method with Vmax=7 obtains the lowest average quality. Therefore, from Table 6 and Figure 19 we can see that the proposed method in offline context with Vmax=8 or 9 can provide the best performance.Figure 19
Buffer level curves of SDP method, bitrate estimation based method, and the proposed method (Max9, Max8, and Max7) in offline context.


## 6.1. Experiment Setup

For the simulation, our test-bed consists of a client running Java 8.0 which implements the adaptation and a server running Apache2 which holds the media segments. The client runs on a Window 7 computer with an Intel i5-1.7 GHz CPU and 4 GB memory and the server runs on Ubuntu 12.04LTS (with default TCP CUBIC) with 1 G RAM. The channel bandwidth is simulated using DummyNet [28]. We use the Tokyo Olympic video from [6]. For the video, Vmax=9, and, for the bandwidth, W=10. Since we measure the buffer size and compute the buffer cost in the segment duration unit, we implement our adaptation method with one setting of the segment duration. In our experiments, we select a segment duration of 2 seconds, which is similar to those of [7, 8, 10]. The impacts of segment durations on adaptation performance have been considered in some recent studies [29, 30]. Further evaluation of different segment durations with a fixed buffer size (in seconds) will be reserved for our future work. Maximum buffer level is set to 5 segments (i.e., 10 seconds) and optimal buffer level is set to 4 segments (i.e., 8 seconds).In our method, a streaming provider can adjust the balance between the requirements for high quality level, preventing video stalling, and reducing quality switches by changing the trade-off parametersα,γ, and β. Since selecting an optimal combination of trade-off parameters of the cost function involves solving a hard optimization problem, it will be investigated in our future work. In this paper, we select qualitatively the trade-off parameters of cost function as follows. Initially, we fix β and select the other two parameters. Since we want to prioritize requirement of smooth quality switch, γ is selected so that the contribution of the quality switch cost Δq is higher than that of buffer cost Δb. With parameter α, because the bitrate cost Δr can be up to thousands, parameter α should be small to reduce the contribution of the bitrate cost to the overall cost C. Based on our experience, good empirical values on parameters α,β, and γ are 0.003, 4, and 20, respectively.


## 6.2. Experimental Results

In the first part of the experiment, we evaluate the accuracy of performance prediction in offline context using a given bandwidth trace obtained from a mobile network [12]. In this context, the number of video segments L is 300.Figures5 and 6 show the predicted performance using the formulas presented in Section 5 and the measured performance obtained from the experiments when the maximum allowed version is set to 7, 8, and 9. These figures point out that the prediction results are close to the measurement ones. We can see from Figure 5 that when the maximum version increases the average version as well as the average version switch also increases. Figure 6 shows that, in both prediction and measurement cases, when the maximum version is 7, the underflow probability is almost zero and increases very slowly when the maximum allowed version increases. This analysis implies that setting the maximum version to 8 is reasonable in terms of balancing between average video quality and quality switch; meanwhile setting the maximum version to 7 ensures a very stable streaming experience.Figure 5
Predicted performance and measured performance in terms of average version and average version switch per segment in offline context using a given bandwidth trace.Figure 6
Predicted performance and measured performance in terms of average buffer and underflow probability in offline context using a given bandwidth trace.Figures7, 8, and 9 show the bitrate and version switch behavior of the proposed method in the three cases of maximum version. It can be seen very clearly from these figures that when the maximum version is reduced, the number of version switches (or quality changes) decreases.Figure 7
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 9.Figure 8
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 8.Figure 9
Experimental results of proposed method in offline context using a given bandwidth trace withVmax = 7.Table1 shows more detailed statistics of the experimental results in three cases of maximum version. It can be drawn from the table that there is no significant difference between the predicted performance and the measured performance.Table 1
Compare predicted performance and measured performance in offline context using a given bandwidth trace.
Statistics
V
m
a
x
=
9
V
m
a
x
=
8
V
m
a
x
=
7
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.48
8.52
9.59
8.77
9.8
9.68
A
q
7.64
7.88
7.45
7.73
6.48
6.87
A
s
w
0.30
0.18
0.14
0.09
0.03
0.02
P
r
u
n
d
f
l
o
w
0.01
0.00
0.006
0.00
0.00
0.00In the second part of the experiment, we use two history bandwidth traces recorded from two previous streaming sessions of the client. The CDFs of both bandwidths are shown in Figure10.Figure 10
The cumulative distribution function of bw1 and bw2.Bandwidthbw1 is used for calculating the statistical models and bw2 is used in the simulation for measuring performance parameters. Figures 11, 12, and 13 show the bitrate and version switch behavior of the proposed method in the three cases of maximum version. The detailed results are listed in Table 2. Based on these figures and table, we affirm once again that the mathematical performance prediction model agrees well with the measurement.Table 2
Compare predicted performance and measured performance in offline context using a history bandwidth trace for statistics.
Statistics
V
m
a
x
=
9
V
m
a
x
=
8
V
m
a
x
=
7
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.52
8.44
9.61
8.85
9.69
9.65
A
q
7.48
7.92
7.27
7.71
6.80
6.98
A
s
w
0.31
0.19
0.18
0.11
0.08
0.05
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Figure 11
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 9.Figure 12
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 8.Figure 13
Experimental results of proposed method in offline context using a history bandwidth trace withVmax = 7.In the third part of the experiment, we consider the online context in which the prediction of the future bandwidth is based on the statistical parameters of all previous segments. Specifically, we divide an entire session into chunks, each of which hasL video segments. We treat each chunk individually as a mini streaming session. Assuming that, initially we have enough statistical data to predict the performance of the first chunk. The prediction of the subsequent chunks will be done based on the previous chunks. At the beginning of each chunk, the bandwidth statistics are updated leading to a recomputation of the policy set and the performance. In the experiment, we set L to 100 video segments. It can be observed from our experiments that it took only several seconds to (re)compute the whole model. Therefore, the computational overhead is about 3%, which is appropriate for the online context. Figures 14, 15, and 16 show the adaptation, bitrate, and version switch behavior of the proposed method in the three cases of maximum version. The predicted performance and the measured performance in the online context in the three cases are presented in detail in Tables 3, 4, and 5. It is very clear that, in the online context, the predicted performance is also very close to the measured performance.Table 3
Compare predicted performance and measured performance in online context withVmax=9.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.31
8.96
8.92
8.45
9.36
8.86
A
q
7.69
8.04
7.89
7.68
7.83
7.72
A
s
w
0.05
0.09
0.04
0.07
0.02
0.05
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Table 4
Compare predicted performance and measured performance in online context withVmax=8.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.53
9.01
9.42
8.63
9.42
8.96
A
q
7.56
7.86
7.65
7.63
7.65
7.67
A
s
w
0.00
0.03
0.01
0.03
0.02
0.04
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Table 5
Compare predicted performance and measured performance in online context withVmax=7.
Statistics
Chunk 1
Chunk 2
Chunk 3
Predicted
Measured
Predicted
Measured
Predicted
Measured
A
b
(
s
)
9.77
9.93
9.77
9.52
9.83
9.59
A
q
6.92
7.00
6.92
6.95
6.92
7.00
A
s
w
0.00
0.00
0.00
0.02
0.00
0.00
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00
0.00Figure 14
Experimental results of the proposed method in online context withVmax = 9.Figure 15
Experimental results of the proposed method in online context withVmax = 8.Figure 16
Experimental results of the proposed method in online context withVmax = 7.Next, we compare our proposed method with the SDP method [10] and the bitrate estimation based method [14] using the same simulation settings as in the first part of our experiment. The experimental results obtained by simulating the SDP method [10] and bitrate estimation based method [14] are shown in Figures 17 and 18, respectively. We can see that both methods provide very fluctuating version switches. The detailed statistics of these adaptation methods with the maximum version set to 9 and our proposed method with the maximum version set to 7, 8, and 9 are shown in Table 6. It can be seen that the performance of the bitrate estimation method is less effective than our method in terms of average version and average version switch. Regarding the SDP method, it is evident that the performance of this method is the worst among all (with low average version and highest average switch per segment). This can be expected because this method was not originally designed for real VBR videos.Table 6
Statistics of different adaptation methods in offline context.
Statistics
SDP methodVmax=9
Bitrate estimation methodVmax=9
Proposed methodVmax=9
Proposed methodVmax=8
Proposed methodVmax=7
A
b
(
s
)
8.62
8.85
8.52
8.77
9.68
A
q
7.40
7.85
7.88
7.73
6.87
A
s
w
0.76
0.43
0.18
0.09
0.02
P
r
u
n
d
f
l
o
w
0.00
0.00
0.00
0.00
0.00Figure 17
Experimental results of SDP method.Figure 18
Experimental results of bitrate estimation based method.The buffer level curves of the three methods in offline context are shown in Figure19. We can see that all buffer curves imply streaming sessions without any freezes for users. Among these, the SDP method and the proposed method with Vmax=9 provide the most unstable buffers, while the bitrate estimation based method and the proposed method with Vmax=7 provide the most stable buffers. It can be observed from Figures 17 and 18 and Table 6 that the SDP method and bitrate estimation based method result in very fluctuating version curves, and the proposed method with Vmax=7 obtains the lowest average quality. Therefore, from Table 6 and Figure 19 we can see that the proposed method in offline context with Vmax=8 or 9 can provide the best performance.Figure 19
Buffer level curves of SDP method, bitrate estimation based method, and the proposed method (Max9, Max8, and Max7) in offline context.


## 7. Conclusion

In this paper, we have proposed an adaptation method for HTTP streaming based on stochastic dynamic programming. The system model was targeted at real bandwidth trace with strong bitrate fluctuation of VBR videos. Furthermore, we have developed a model to predict the system performance with the aim of choosing the best setting based on the performance requirements. The experimental results have shown that our method can effectively adapt VBR videos and perform accurate performance prediction which is useful in planning adaptation policy.



---

*Source: 7323681-2017-07-02.xml*