# Learning-Based Online QoE Optimization in Multi-Agent Video Streaming
## 3 System Model
### Class 2: Users Continuing with the Old Video  
We now discuss the stall time model for users who continue the video from time slot \(l-1\) to time slot \(l\) (Figure 3). Assume that, in the previous time slot \(l-1\), the total slot duration is \(T^{\prime}\). At the moment of \(T^{\prime}\), a chunk--denoted by \(0\)--is being downloaded. As a result of the chunks that were continuously downloaded, by evaluating \(T^{\prime}\) and the download speed \(d_{k}(l-1)\), we can calculate the length or ratio of chunk \(0\), which has not been downloaded by  
\[l_{0}=1-d_{k}(l-1)\cdot\bigg{(}T^{\prime}\bmod\frac{1}{d_{k}(l-1)}\bigg{)}. \tag{9}\]  
Note that, since the length of the chunks are normalized, we have: \(l_{m}=1,\forall m\
eq 0\). Downloaded with speed \(d_{k}(l)\), the leftover chunk \(0\) with length \(l_{0}\) needs time \(l_{0}/d_{k}(l)\) to be ready for the user to play it. Following the continuous downloading rule, in time slot \(l\), we have \(t^{\prime}(1)=l_{0}/d_{k}(l)\). Then, similar to Equation (1), the rest of the \(t^{\prime}(m)\) can be recursively obtained. \[t^{\prime}(m)=\left\{ \begin{array}{ll}t^{\prime}(m-1)+\frac{1}{d_{k}(l)},&m> 1,\\ \frac{l_{0}}{d_{k}(l)},&m=1,\\ 0,&m=0.\end{array} \right. \tag{10}\]  
We denote the last chunk being played in time slot \(l-1\) as chunk \(-n\), which is the video chunk ahead of chunk \(0\) by \(n\), and we denote its finish time calculated in the previous time slot by \(\bar{t}^{\prime}(-n)\). If \(n=1\) and \(\bar{t}^{\prime}(-1)\leq T^{\prime}\), we know that all chunks before chunk \(0\) have finished playing in slot \(l-1\). Otherwise, chunk \(-n\) is being played half-way at the moment of the time slot transition. For the latter case, user \(k\) will continue the play of video chunk \(-n\) at the beginning of time slot \(l\). Then, in the new time slot \(l\), because the video bitrate is not changed, chunk \(-n\) will be finished at \(\bar{t}(-n)=\bar{t}^{\prime}(-n)-T^{\prime}\). Since, at the beginning of slot \(l\), chunk \(0\) is being downloaded, we know that chunks in interval \(\{-n,\cdots,-1\}\)  
Figure 3: The stall time calculation for users who continue with the video from previous time slot. in \(t\) are all ready to be played. Therefore, we can derive the play finish time of chunks \(\{-n,\cdots,-1\}\) in slot \(l\) as:  
\[\bar{t}(m)=\left\{ \begin{array}{ll}\bar{t}(m-1)+\frac{1}{\tau_{k}(l-1)},&-n<m \leq-1,\\ \bar{t}^{\prime}(m)-T^{\prime},&m=-n.\end{array} \right. \tag{11}\]  
As the download finish time \(t^{\prime}(1)\) and play finish time \(\bar{t}(-1)\) are defined, the leftover video chunk \(0\) is played at time \(t(0)=\max(t^{\prime}(1),\bar{t}(-1))\) and finished at \(\bar{t}(0)=t(0)+1/\tau_{k}(l)\). With all of the leftover chunk issues tackled, we finally obtain the chunk time equations for time slot \(l\):  
\[t^{\prime}(m)=\left\{ \begin{array}{ll}t^{\prime}(m-1)+\frac{1}{d_{k}(l)},&m> 1,\\ \frac{l_{0}}{d_{k}(l)},&m=1,\end{array} \right. \tag{12}\]  
\[t(m)=\max(t^{\prime}(m+1),\bar{t}(m-1)),m\geq 0, \tag{13}\]  
\[\bar{t}(m)=\left\{ \begin{array}{ll}t(m)+\frac{1}{\tau_{k}(l)},&m>0,\\ t(m)+\frac{1}{\tau_{k}(l-1)},&0\geq m>-n,\\ \bar{t}^{\prime}(m)-\bar{T}^{\prime},&m=-n.\end{array} \right. \tag{14}\]  
With all of the time equations obtained, we can now calculate the stall time using the similar procedure shown in the previous sub-section. Since, for \(m<0\), all of the chunks are being played stall-free, if the slot ends at time \(T<\bar{t}(-1)\), the stall time will be zero. From chunk \(m>0\), it is possible that the stall appears between the gap where chunk \(m-1\) is finished, while chunk \(m\) is not downloaded yet (\(\bar{t}(m-1)<t(m)=t^{\prime}(m+1)\)). If \(T\) happens to be in this gap, the stall time \(T_{s}(T)\) will be the accumulated waiting time of chunks \([0,m-1]\) (denoted as \(T_{s}(t(m-1))\)) plus the additional time between \(T\) and \(\bar{t}(m-1)\).