## 3 Comparisons of Actor-Critic Frameworks
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Type** & **Ref.** & **Optimization Objective** & **Agent** & **State Space** & **Action Space** \\ \hline \multirow{8}{*}{Solely FCN or MLP-based} & \multirow{4}{*}{[45]} & To minimize total cost consisting of energy consumption and delay-aware task priority-based utility in UAVSNs assisted edge computing mission for GUs & \multirow{4}{*}{3D position of UAVs and task attributes} & Binary task offloading, transmit power of GUs, UAV position, and computing resource allocation at UAVs \\ \cline{2-4}  & \multirow{4}{*}{[47]} & To maximize energy efficiency and throughput for GU2U links with energy harvesting at UAVs via beamforming from terrestrial base station & UAV movement \\ \cline{2-4}  & & & & \\ \cline{2-4}  & & & & \\ \hline \multirow{8}{*}{RNN-based} & \multirow{4}{*}{[48]} & To minimize average Ao in UAVSNs assisted data collection mission from GUs & \multirow{4}{*}{UAVs} & 2D position of UAVs and AoI of GUs & Scheduling data transmission for GU and UAV mobility \\ \cline{2-4}  & \multirow{4}{*}{[49]} & To minimize packet routing latency in the end-to-end path while avoiding network congestion & \multirow{4}{*}{UAVs} & Location, ID number, and queue length of neighboring UAVs, and location of destination packet & Next hop selection from local neighbor list \\ \cline{2-4}  & \multirow{4}{*}{[24]} & To plant trajectory for UAVSNs while maintaining formation in an unknown dense obstacle & Leader and follower UAVs & 3D position of UAVs, pitch and yaw angle, and relative distances with nearest obstacles & Acceleration of UAVs \\ \hline \multirow{8}{*}{[2]} & \multirow{8}{*}{To maximize a multi-metric link utility to route data packets in UAVSNs by utilizing two-hop neighbor information} & \multirow{4}{*}{UAVs} & Virtual forces, binary frequency band paring matrix and link duration up to two hops, residual energy, and queue & UAV trajectory control, frequency band, and next-hop selection \\ \cline{2-4}  & \multirow{4}{*}{[1]} & To minimize energy consumption of UAVs and delay in air-ground integrated edge computing networks & \multirow{4}{*}{UAVs} & Virtual forces, SINR, GU-UAVs association matrix, task attributes, queue size, and remaining energy of UAVs & Trajectory of UAVs, UAV-GUs association, bandwidth, and computing resource allocation \\ \cline{2-4}  & \multirow{4}{*}{[10]} & To maximize throughput for GU links to provide communication service & \multirow{4}{*}{UAVs} & For GUs: GU and UAV position, and and GU transmit power for UAVs: UAV position, and GU-UAVs paring matrix & UAVs: trajectory planning and bandwidth allocation, and GUs: binary UAV-GUs association \\ \cline{2-4}  & \multirow{4}{*}{[49]} & To minimize target searching time and maximize the area coverage while avoiding collision avoidance & \multirow{4}{*}{UAVs} & Probability map of target, visiting time, neighboring UAVs and threats, and location of UAVs from a discrete set \\ \cline{2-4}  & \multirow{4}{*}{[5]} & To minimize AoI for computing stochastic tasks in a multi-agent mobile GUs-UAV-cloud environment & \multirow{4}{*}{GUs and UAVs} & For GUs: its position and queue backlog size, and available bandwidth & Computation offloading, UAV trajectory, and bandwidth allocation \\ \cline{2-4}  & \multirow{4}{*}{[52]} & To minimize long-term system costs consisting of energy and delay in a mobile device–edge server–cloud networks & \multirow{4}{*}{GUs} & Remaining computing resource, queue backlog size, channel gain, GUs position, and task attributes & Binary task offloading, computing, and bandwidth allocation \\ \cline{2-4}  & \multirow{4}{*}{[53]} & To minimize the task execution latency for GUs in UAVs by jointly controlling UAV movement and partial task offloading ratio & \multirow{4}{*}{GUs and UAVs} & For GUs: position of GUs, task attributes, and channel state and UAV power control from UAV \\ \cline{2-4}  & \multirow{4}{*}{[53]} & To minimize long-term average weighted task execution delay and energy consumption for multi-UAV and multi-edge server networks & \multirow{4}{*}{UAVs} & Task attributes and distance between UAVs & Binary task offloading, transmit power, bandwidth, and computing resources \\ \hline \multirow{2}{*}{QNN-based} & \multirow{2}{*}{[55]} & To maximize data rate for U2 GU links under Q6S constraints & \multirow{2}{*}{UAVs} & GU-UAV paring matrix, and QoS requirement & UAVs movement \\ \cline{2-4}  & & & & \\ \hline \hline \end{tabular}
\end{table}