# Learning-Based Online QoE Optimization in Multi-Agent Video Streaming
## 3 System Model
### Class 1: User Requests a New Video  
We first calculate the stall durations for user \(k\) that has requested a video change. As shown in Figure 2, user \(k\) starts to fetch a video from the beginning of the slot \(l\). We assume that the chunk \(m\) is played in time-slot \(l\); if not, the calculations for the stall duration for those chunks will be studied in Case 2. With the given downloading speed and bitrate, we can observe the relationships between \(t^{\prime}(l,m)\), \(t(l,m)\), and \(\bar{t}(l,m)\):  
\[t^{\prime}_{k}(l,m)=\left\{\begin{array}{ll}t^{\prime}_{k}(l,m-1)+\frac{1}{d _{k}(l)},&m>1,\\ 0,&m=1,\end{array}\right. \tag{1}\]  
\[t_{k}(l,m)=\left\{\begin{array}{ll}\max(t^{\prime}_{k}(l,m),\bar{t}_{k}(l,m-1 )),&m>1,\\ t^{\prime}_{k}(l,m),&m=1,\end{array}\right. \tag{2}\]  
\[\bar{t}_{k}(l,m)=t_{k}(l,m)+\frac{1}{r_{k}(l)},m\geq 1. \tag{3}\]  
Since we will be limiting the analysis for user \(k\) in slot \(l\), we drop the subscript \(k\) and the argument \(l\) from \(t_{k}(l,m),t^{\prime}_{k}(l,m)\), and \(\bar{t}_{k}(l,m)\) for brevity. Let \(T\) denote the time elapsed in time slot \(l\), and let \(T_{s}(l,T,k)\) denote the stall time in slot \(l\) till elapsed time \(T\) for user \(k\). Clearly, when the video downloading speed is equal to or higher than the video  
Figure 2: The stall time calculation for users request a new video triggering the system to move to slot \(l\).  
bitrate (\(d_{k}(l)\geq r_{k}(l)\)), the user only has to wait for the first chunk to arrive, and then experience a stall-free video playback. Otherwise, for \(d_{k}(l)<r_{k}(l)\), three conditions need to be considered for \(T\). If \(T\) is smaller than or equal to \(t(1)\), no video chunk has been played and the stall time is exactly the time elapsed \(T\) in the time slot. Otherwise, if \(T\) lands in an interval in which a video chunk \(m\) is being played, the stall time of \(T\) is equal to the stall time of \(t(m)\), and if \(T\) lands in an interval where the user is waiting for the chunk \(m+1\) to be downloaded, the stall time needs an additional wait after the \(m\)-th chunk is played. Hence, the stall time until the end of the slot \(l\), \(T_{s}(l,T,k)\), can be defined as a recursive conditional function,  
\[T_{s}(l,T,k)=\left\{\begin{array}{ll}T_{s}(l,t(m),k),&t(m)<T\leq\bar{t}(m), \\ &d_{k}(l)<r_{k}(l),m\geq 1,\\ T-\bar{t}(m)+\bar{t}(m)<T\leq t(m+1),\\ T_{s}(l,t(m),k),&d_{k}(l)<r_{k}(l),m\geq 1,\\ \min(T,t(1)),&\text{otherwise}.\end{array}\right. \tag{4}\]  
In the condition of \(d_{k}(l)<r_{k}(l)\), the stall time before the \(m\)-th chunk is downloaded is the key to obtaining the stall time of \(T\). The stall time of \(t(m)\) fits the second condition of Equation (4). Thus, we have:  
\[T_{s}(l,t(m),k)=T_{s}(l,t(m-1),k)+t(m)-\bar{t}(m-1),m>1. \tag{5}\]  
According to Equations (1)-(3), the difference between \(t(m)\) and \(\bar{t}(m-1)\) can be calculated. Thus, Equation (5) can be written in a recursive form:  
\[T_{s}(l,t(m),k)=\left\{\begin{array}{ll}T_{s}(l,t(m-1),k)+\frac{1}{d_{k}(l) }-\frac{1}{r_{k}(l)},&m>1,\\ \frac{1}{d_{k}(l)},&m=1,\end{array}\right. \tag{6}\]  
and further solved as:  
\[T_{s}(l,t(m),k)=\frac{m}{d_{k}(l)}-\frac{m-1}{r_{k}(l)},m\geq 1. \tag{7}\]  
Finally, substitute \(T_{s}(t_{k}(l,m))\) into Equation (4); the stall time of time slot length \(T\) is  
\[T_{s}(l,T,k)=\left\{\begin{array}{ll}\frac{m}{d_{k}(l)}-\frac{m-1}{r_{k}(l) },&t(m)<T\leq\bar{t}(m),\\ &d_{k}(l)<r_{k}(l),m\geq 1,\\ T-\frac{m}{r_{k}(l)},&\bar{t}(m)<T\leq t(m+1),\\ &d_{k}(l)<r_{k}(l),m\geq 1,\\ \min(T,t(1)),&\text{otherwise}.\end{array}\right. \tag{8}\]  
Note that, if some other user \(k^{\prime}\
eq k\) requests a new video triggering an increment in the time slot from \(l\) to \(l+1\), the stall duration analysis will fall to the second class of users. We discuss the stall duration for the second class of users in the next section.