## 3 Problem Formulation and Model Design
### Problem Formulation  
As shown in Figure 1, there are distributed edge servers and mobile users in the edge network video streaming system. Each end user has different trajectories under specific modes of transportation, so the network connection status with adjacent edge nodes is constantly changing. As mentioned earlier, every edge server with limited resources needs to adapt to various requests online in real time, which is called the EUA problem. At the same time, each mobile user also needs to choose an appropriate bitrate based on their local network connection quality to ensure smooth playback, which is known as the bitrate selection problem. Although these two problems aim to achieve different strategies, such as server allocation or bitrate selection, their goals are similar, namely, to maximize users' QoE with limited infrastructure resources or minimizing system costs. \begin{table}
\begin{tabular}{c l} \hline \hline
**Notation** & **Description** \\ \hline \(E=\{e_{1},e_{2},\cdots,e_{i}\}\) & Finite set of edge servers \(e_{i}\), where \(i=1,2,\cdots,m\) \\ \(\mathcal{U}_{i}=\{u_{1},u_{2},\cdots,u_{j}\}\) & Finite set of users \(u_{j}\) under the coverage of the edge server \(e_{i}\) \\ \(C_{i}\) & The set of users set served by the edge server \(e_{i}\), \(C_{i}\subseteq\mathcal{U}_{i}\) \\ \(Con_{i}\) & The maximum number of connections to the edge server \(e_{i}\) \\ \(d_{ij}\) & Geographic distance between the edge server \(e_{i}\) and user \(u_{j}\) \\ \(rtt_{i}(u_{j})\) & The RTT value of the user \(u_{j}\) with the corresponding server \(e_{i}\) \\ \(th_{i}(u_{j})\) & Throughput rate of the user \(u_{j}\) under the coverage of the edge server \(e_{i}\) \\ \(b_{i}(u_{j})\) & The video resolution assigned to the user \(u_{j}\) \\ \(lb_{i}(u_{j})\) & The last video resolution assigned to the user \(u_{j}\) \\ \(ap(|e_{i}|)\) & Decide whether to penalize the reward based on the number of user sets \\  & assigned to the edge server \(e_{i}\) \\ \(tp(th_{i}(u_{j}),b_{i}(u_{j}))\) & Decide whether to penalize the reward based on the \(th_{i}(u_{j})\) and \(b_{i}(u_{j})\) of the user \(u_{j}\) under the coverage of the edge server \(e_{i}\) \\ \(QoE(u_{j})\) & QoE of the user \(u_{j}\) \\ \(EN\) & Number of leased servers \\ \(S_{ij}\) & State information between the server \(e_{i}\) and user \(u_{j}\) \\ \(S_{i}\) & State vector of the server \(e_{i}\), consisting of the status \(S_{ij},j\in U_{i}\) and \(|C_{i}|\) \\ \(\mathcal{S}\) & A state matrix of all server states \\  & The action of the server \(e_{i}\) to the user \(u_{j}\), which is a 2-tuple where the first \\ \(A_{ij}\) & element is the server index and the second element is the assigned \\  & resolution \(b_{i}(u_{j})\) \\ \(A_{i}\) & Action vector between the server \(e_{i}\) and its covered users \(U_{i}\) \\ \(R_{i}\) & Reward for the edge server \(e_{i}\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Notation. Given a video streaming system with distributed edge servers and mobile users, their interaction process is shown in Figure 2 under the AMARL framework. Let us denote the set of edge servers as \(E=\{e_{1},e_{2},\cdots,e_{i}\}\), the users set under the coverage of \(e_{i}\) as \(U_{i}=\{u_{1},u_{2},\cdots,u_{j}\}\), the users set served by \(e_{i}\) as \(C_{i}(C_{i}\subset U_{i})\), and the maximum number of connections to \(e_{i}\) as \(Con_{i}\). Denote geographic distance, round-trip time (RTT), and throughput rate between \(e_{i}\) and \(u_{j}\) as \(d_{ij}\), \(rt_{i}(u_{j})\), and \(th_{i}(u_{j})\), respectively. \(b_{i}(u_{j})\) and \(lb_{i}(u_{j})\) represent the current and last video bitrate assigned to \(u_{j}\). Then the users' \(QoE\) can be defined as  
\[QoE(u_{j}) =bw\times b_{i}(u_{j})-sw\times|b_{i}(u_{j})-lb_{i}(u_{j})|+ \tag{1}\] \[tw\times tp(th_{i}(u_{j}),b_{i}(u_{j}))-rw\times rt_{i}(u_{j}).\]  
Note that the absolute value of \(b_{i}(u_{j})-lb_{i}(u_{j})\) is used to reflect the smoothness of the video bitrate change.